[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:50:58 dist-test-slave-jlzn krb5kdc[27219](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:50:58 dist-test-slave-jlzn krb5kdc[27219](info): set up 2 sockets
Aug 08 03:50:58 dist-test-slave-jlzn krb5kdc[27219](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 03:50:59.392472 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.004s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:50:59 dist-test-slave-jlzn krb5kdc[27219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:50:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:50:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:50:59.576241 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:45413
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:38933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:45413
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:50:59.903795 27235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:50:59.904125 27235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:50:59.904256 27235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:50:59.911278 27235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:50:59.911375 27235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:50:59.911471 27235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:50:59.911561 27235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:50:59.911626 27235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:50:59.921156 27235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:38933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:45413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:45413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:50:59.922562 27235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:50:59.923929 27235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:50:59.936373 27244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:50:59.936429 27245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:50:59.936462 27247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:50:59.936991 27235 server_base.cc:1034] running on GCE node
I20260808 03:50:59.937824 27235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:50:59.939210 27235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:50:59.940412 27235 hybrid_clock.cc:648] HybridClock initialized: now 1786161059940354 us; error 39 us; skew 500 ppm
I20260808 03:50:59.940733 27235 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:50:59 dist-test-slave-jlzn krb5kdc[27219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161059, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:50:59.943663 27235 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:50:59.945063 27235 webserver.cc:460] Webserver started at http://127.26.146.190:43805/ using document root <none> and password file <none>
I20260808 03:50:59.945959 27235 fs_manager.cc:359] Metadata directory not provided
I20260808 03:50:59.946069 27235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:50:59.946516 27235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:50:59.950109 27235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "82be96bf9415474eb85532e8f41da9f0"
format_stamp: "Formatted at 2026-08-08 03:50:59 on dist-test-slave-jlzn"
server_key: "3a40a87cf34eb62aeba3af0b300e2f2e"
server_key_iv: "3bfb33ebf4a5f407b92c4d2427f42bad"
server_key_version: "encryptionkey@0"
I20260808 03:50:59.950814 27235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "82be96bf9415474eb85532e8f41da9f0"
format_stamp: "Formatted at 2026-08-08 03:50:59 on dist-test-slave-jlzn"
server_key: "3a40a87cf34eb62aeba3af0b300e2f2e"
server_key_iv: "3bfb33ebf4a5f407b92c4d2427f42bad"
server_key_version: "encryptionkey@0"
I20260808 03:50:59.957566 27235 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260808 03:50:59.962746 27256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:50:59.964614 27235 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260808 03:50:59.964792 27235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "82be96bf9415474eb85532e8f41da9f0"
format_stamp: "Formatted at 2026-08-08 03:50:59 on dist-test-slave-jlzn"
server_key: "3a40a87cf34eb62aeba3af0b300e2f2e"
server_key_iv: "3bfb33ebf4a5f407b92c4d2427f42bad"
server_key_version: "encryptionkey@0"
I20260808 03:50:59.964962 27235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:50:59.977941 27235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:50:59.981943 27235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:50:59.982201 27235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:50:59.999756 27235 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:45413
I20260808 03:50:59.999776 27330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:45413 every 8 connection(s)
I20260808 03:51:00.001170 27235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:51:00.002243 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 27235
I20260808 03:51:00.002552 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:51:00.004645 27210 external_mini_cluster.cc:1428] Setting key 106a8256d9649c00c18985211a240504
I20260808 03:51:00.005592 27331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:00.017907 27331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82be96bf9415474eb85532e8f41da9f0: Bootstrap starting.
I20260808 03:51:00.021741 27331 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82be96bf9415474eb85532e8f41da9f0: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:00.022943 27331 log.cc:824] T 00000000000000000000000000000000 P 82be96bf9415474eb85532e8f41da9f0: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:00.026436 27331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82be96bf9415474eb85532e8f41da9f0: No bootstrap required, opened a new log
Aug 08 03:51:00 dist-test-slave-jlzn krb5kdc[27219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:00.035995 27331 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82be96bf9415474eb85532e8f41da9f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82be96bf9415474eb85532e8f41da9f0" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 45413 } }
I20260808 03:51:00.036346 27331 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82be96bf9415474eb85532e8f41da9f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:00.036470 27331 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82be96bf9415474eb85532e8f41da9f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82be96bf9415474eb85532e8f41da9f0, State: Initialized, Role: FOLLOWER
I20260808 03:51:00.037238 27331 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82be96bf9415474eb85532e8f41da9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82be96bf9415474eb85532e8f41da9f0" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 45413 } }
I20260808 03:51:00.037532 27331 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82be96bf9415474eb85532e8f41da9f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:51:00.037685 27331 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82be96bf9415474eb85532e8f41da9f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:51:00.037882 27331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82be96bf9415474eb85532e8f41da9f0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:00.040755 27331 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82be96bf9415474eb85532e8f41da9f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82be96bf9415474eb85532e8f41da9f0" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 45413 } }
I20260808 03:51:00.041555 27331 leader_election.cc:302] T 00000000000000000000000000000000 P 82be96bf9415474eb85532e8f41da9f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82be96bf9415474eb85532e8f41da9f0; no voters:
I20260808 03:51:00.042275 27331 leader_election.cc:288] T 00000000000000000000000000000000 P 82be96bf9415474eb85532e8f41da9f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:51:00.042459 27338 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82be96bf9415474eb85532e8f41da9f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:00.043071 27338 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82be96bf9415474eb85532e8f41da9f0 [term 1 LEADER]: Becoming Leader. State: Replica: 82be96bf9415474eb85532e8f41da9f0, State: Running, Role: LEADER
I20260808 03:51:00.043650 27338 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82be96bf9415474eb85532e8f41da9f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82be96bf9415474eb85532e8f41da9f0" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 45413 } }
I20260808 03:51:00.044308 27335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:00.008470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56356 (local address 127.26.146.190:45413)
0808 03:51:00.009263 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:00.009274 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:00.010468 (+ 1194us) server_negotiation.cc:407] Connection header received
0808 03:51:00.019060 (+ 8592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:00.019088 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:00.019424 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:00.019766 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:00.022535 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:00.024175 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:00.026641 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:00.027311 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:00.027431 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:00.034617 (+ 7186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:00.034632 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:00.034645 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:00.034686 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:00.038426 (+ 3740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:00.039184 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:00.039193 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:00.039200 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:00.039414 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:00.039874 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:00.039879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:00.039883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:00.040574 (+ 691us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:00.040907 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:00.041756 (+ 849us) server_negotiation.cc:299] Negotiation successful
0808 03:51:00.042176 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":351,"threads_started":1}
I20260808 03:51:00.045859 27331 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82be96bf9415474eb85532e8f41da9f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:51:00.046701 27340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82be96bf9415474eb85532e8f41da9f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82be96bf9415474eb85532e8f41da9f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82be96bf9415474eb85532e8f41da9f0" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 45413 } } }
I20260808 03:51:00.047070 27340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82be96bf9415474eb85532e8f41da9f0 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:00.049149 27339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82be96bf9415474eb85532e8f41da9f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82be96bf9415474eb85532e8f41da9f0. Latest consensus state: current_term: 1 leader_uuid: "82be96bf9415474eb85532e8f41da9f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82be96bf9415474eb85532e8f41da9f0" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 45413 } } }
I20260808 03:51:00.049451 27339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82be96bf9415474eb85532e8f41da9f0 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:00.053110 27347 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:51:00.058969 27347 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:51:00.068269 27347 catalog_manager.cc:1347] Generated new cluster ID: 1873dfdd726646d79ca3fdcc679c6be7
I20260808 03:51:00.068379 27347 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:51:00.089480 27347 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:51:00.090891 27347 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:51:00.101120 27347 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82be96bf9415474eb85532e8f41da9f0: Generated new TSK 0
I20260808 03:51:00.101904 27347 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
I20260808 03:51:00.153507 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:45413
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:38933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:00.555584 27368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:00.555886 27368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:00.556041 27368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:00.563154 27368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:00.563259 27368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:00.563413 27368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:51:00.573333 27368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:38933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:45413
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:00.574651 27368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:00.576028 27368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:00.588557 27373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:00.588586 27374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:00.588511 27377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:00.589352 27368 server_base.cc:1034] running on GCE node
I20260808 03:51:00.589908 27368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:00.590788 27368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:00.591982 27368 hybrid_clock.cc:648] HybridClock initialized: now 1786161060591933 us; error 37 us; skew 500 ppm
I20260808 03:51:00.592289 27368 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:00 dist-test-slave-jlzn krb5kdc[27219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161060, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:00.595283 27368 init.cc:377] Logged in from keytab as kudu/127.26.146.129@KRBTEST.COM (short username kudu)
I20260808 03:51:00.596625 27368 webserver.cc:460] Webserver started at http://127.26.146.129:38303/ using document root <none> and password file <none>
I20260808 03:51:00.597260 27368 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:00.597378 27368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:00.597669 27368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:00.600531 27368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "c0a116363be04a599ecc6253acbf4af8"
format_stamp: "Formatted at 2026-08-08 03:51:00 on dist-test-slave-jlzn"
server_key: "c6bcdfe8bdb8882f7611b0c97790ab29"
server_key_iv: "2d654ebe78b5cc46729a90b705b042c7"
server_key_version: "encryptionkey@0"
I20260808 03:51:00.601357 27368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "c0a116363be04a599ecc6253acbf4af8"
format_stamp: "Formatted at 2026-08-08 03:51:00 on dist-test-slave-jlzn"
server_key: "c6bcdfe8bdb8882f7611b0c97790ab29"
server_key_iv: "2d654ebe78b5cc46729a90b705b042c7"
server_key_version: "encryptionkey@0"
I20260808 03:51:00.606596 27368 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 03:51:00.609894 27385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:00.611160 27368 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260808 03:51:00.611320 27368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "c0a116363be04a599ecc6253acbf4af8"
format_stamp: "Formatted at 2026-08-08 03:51:00 on dist-test-slave-jlzn"
server_key: "c6bcdfe8bdb8882f7611b0c97790ab29"
server_key_iv: "2d654ebe78b5cc46729a90b705b042c7"
server_key_version: "encryptionkey@0"
I20260808 03:51:00.611523 27368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:00.626538 27368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:00.630509 27368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:00.630779 27368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:00.631739 27368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:00.633097 27368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:00.633177 27368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:00.633291 27368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:00.633347 27368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:00.667125 27368 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:43261
I20260808 03:51:00.667158 27555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:43261 every 8 connection(s)
I20260808 03:51:00.668426 27368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:51:00.673821 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 27368
I20260808 03:51:00.674026 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:51:00.674618 27210 external_mini_cluster.cc:1428] Setting key ec96f5c29792a2055c3b9ae35dba8103
Aug 08 03:51:00 dist-test-slave-jlzn krb5kdc[27219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161060, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:00.687347 27558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:00.670945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:51593 (local address 127.26.146.190:45413)
0808 03:51:00.671420 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:00.671426 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:00.672395 (+ 969us) server_negotiation.cc:407] Connection header received
0808 03:51:00.674171 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:00.674177 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:00.674253 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:00.674351 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:00.675847 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:00.676568 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:00.678086 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:00.678491 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:00.678608 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:00.681937 (+ 3329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:00.681941 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:00.681944 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:00.681975 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:00.684414 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:00.685100 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:00.685104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:00.685106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:00.685229 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:00.685657 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:00.685660 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:00.685662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:00.685925 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:00.686056 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:00.686758 (+ 702us) server_negotiation.cc:299] Negotiation successful
0808 03:51:00.686988 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":217,"threads_started":1}
I20260808 03:51:00.688445 27560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:00.671513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:45413 (local address 127.26.146.129:51593)
0808 03:51:00.672245 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:00.672288 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:00.673876 (+ 1588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:00.674588 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:00.674595 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:00.675114 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:00.675660 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:00.675678 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:00.676740 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:00.676744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:00.677951 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:00.677964 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:00.678715 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:00.678723 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:00.678952 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:00.679594 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:00.679616 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:00.681733 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:00.684544 (+ 2811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:00.684553 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:00.684570 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:00.684984 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:00.685332 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:00.685337 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:00.685341 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:00.685544 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:00.686046 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:00.686053 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:00.686395 (+ 342us) client_negotiation.cc:769] Sending connection context
0808 03:51:00.686627 (+ 232us) client_negotiation.cc:240] Negotiation successful
0808 03:51:00.687143 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":306,"threads_started":1}
I20260808 03:51:00.690248 27556 heartbeater.cc:347] Connected to a master server at 127.26.146.190:45413
I20260808 03:51:00.690531 27556 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:00.691264 27556 heartbeater.cc:511] Master 127.26.146.190:45413 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.26.146.130@KRBTEST.COM; defaulting to no policy
I20260808 03:51:00.693420 27275 ts_manager.cc:194] Registered new tserver with Master: c0a116363be04a599ecc6253acbf4af8 (127.26.146.129:43261)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.130@KRBTEST.COM" created.
I20260808 03:51:00.695148 27275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.129@KRBTEST.COM'} at 127.26.146.129:51593
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
I20260808 03:51:00.716782 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:45413
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:38933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:01.197582 27564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:01.197851 27564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:01.197994 27564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:01.205343 27564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:01.205447 27564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:01.205601 27564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:51:01.215145 27564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:38933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:45413
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:01.216558 27564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:01.217846 27564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:01.230500 27570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:01.230530 27571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:01.230471 27574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:01.231350 27564 server_base.cc:1034] running on GCE node
I20260808 03:51:01.232025 27564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:01.232909 27564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:01.234119 27564 hybrid_clock.cc:648] HybridClock initialized: now 1786161061234079 us; error 30 us; skew 500 ppm
I20260808 03:51:01.234406 27564 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:01 dist-test-slave-jlzn krb5kdc[27219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161061, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:01.237573 27564 init.cc:377] Logged in from keytab as kudu/127.26.146.130@KRBTEST.COM (short username kudu)
I20260808 03:51:01.238932 27564 webserver.cc:460] Webserver started at http://127.26.146.130:39587/ using document root <none> and password file <none>
I20260808 03:51:01.239553 27564 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:01.239652 27564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:01.239966 27564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:01.242808 27564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "fafb017cd6074204ae1937f508cbdeb3"
format_stamp: "Formatted at 2026-08-08 03:51:01 on dist-test-slave-jlzn"
server_key: "b579fc253f6909e352baaeb3f5c47ea4"
server_key_iv: "64df2ac1596349efd86c9b651b345b79"
server_key_version: "encryptionkey@0"
I20260808 03:51:01.243659 27564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "fafb017cd6074204ae1937f508cbdeb3"
format_stamp: "Formatted at 2026-08-08 03:51:01 on dist-test-slave-jlzn"
server_key: "b579fc253f6909e352baaeb3f5c47ea4"
server_key_iv: "64df2ac1596349efd86c9b651b345b79"
server_key_version: "encryptionkey@0"
I20260808 03:51:01.248993 27564 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260808 03:51:01.252441 27584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:01.253757 27564 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:51:01.253894 27564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "fafb017cd6074204ae1937f508cbdeb3"
format_stamp: "Formatted at 2026-08-08 03:51:01 on dist-test-slave-jlzn"
server_key: "b579fc253f6909e352baaeb3f5c47ea4"
server_key_iv: "64df2ac1596349efd86c9b651b345b79"
server_key_version: "encryptionkey@0"
I20260808 03:51:01.254063 27564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:01.283445 27564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:01.287302 27564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:01.287554 27564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:01.288498 27564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:01.289873 27564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:01.289947 27564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:01.290040 27564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:01.290091 27564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:01.320996 27564 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:39201
I20260808 03:51:01.321039 27749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:39201 every 8 connection(s)
I20260808 03:51:01.322276 27564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:51:01.324815 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 27564
I20260808 03:51:01.325026 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:51:01.325585 27210 external_mini_cluster.cc:1428] Setting key 9f53d60f154323c978908499dfee548e
Aug 08 03:51:01 dist-test-slave-jlzn krb5kdc[27219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161061, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:01.342962 27752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:01.324923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:40777 (local address 127.26.146.190:45413)
0808 03:51:01.325398 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:01.325402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:01.326578 (+ 1176us) server_negotiation.cc:407] Connection header received
0808 03:51:01.328802 (+ 2224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:01.328807 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:01.328884 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:01.328976 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:01.330532 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:01.331640 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:01.333227 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:01.333693 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:01.333813 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:01.337218 (+ 3405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:01.337223 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:01.337226 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:01.337259 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:01.339901 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:01.340580 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:01.340586 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:01.340590 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:01.340723 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:01.341163 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:01.341169 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:01.341173 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:01.341531 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:01.341697 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:01.342323 (+ 626us) server_negotiation.cc:299] Negotiation successful
0808 03:51:01.342552 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":253,"threads_started":1}
I20260808 03:51:01.343683 27753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:01.325587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:45413 (local address 127.26.146.130:40777)
0808 03:51:01.326419 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:01.326467 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:01.328547 (+ 2080us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:01.329233 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:01.329242 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:01.329736 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:01.330297 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:01.330322 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:01.331841 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:01.331847 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:01.333084 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:01.333097 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:01.333917 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:01.333925 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:01.334146 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:01.334779 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:01.334802 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:01.336999 (+ 2197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:01.340037 (+ 3038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:01.340046 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:01.340059 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:01.340461 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:01.340822 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:01.340827 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:01.340831 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:01.341014 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:01.341649 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:01.341654 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:01.341956 (+ 302us) client_negotiation.cc:769] Sending connection context
0808 03:51:01.342172 (+ 216us) client_negotiation.cc:240] Negotiation successful
0808 03:51:01.342552 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":325,"threads_started":1}
I20260808 03:51:01.345026 27750 heartbeater.cc:347] Connected to a master server at 127.26.146.190:45413
I20260808 03:51:01.345314 27750 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.26.146.131@KRBTEST.COM; defaulting to no policy
I20260808 03:51:01.346104 27750 heartbeater.cc:511] Master 127.26.146.190:45413 requested a full tablet report, sending...
I20260808 03:51:01.347575 27275 ts_manager.cc:194] Registered new tserver with Master: fafb017cd6074204ae1937f508cbdeb3 (127.26.146.130:39201)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.131@KRBTEST.COM" created.
I20260808 03:51:01.348443 27275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.130@KRBTEST.COM'} at 127.26.146.130:40777
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
I20260808 03:51:01.370507 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:45413
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:38933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:51:01.698781 27556 heartbeater.cc:503] Master 127.26.146.190:45413 was elected leader, sending a full tablet report...
W20260808 03:51:01.737321 27758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:01.737646 27758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:01.737785 27758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:01.745201 27758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:01.745327 27758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:01.745508 27758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:51:01.756127 27758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:38933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:45413
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:01.757400 27758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:01.758853 27758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:01.772370 27764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:01.772467 27765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:01.772583 27767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:01.773568 27758 server_base.cc:1034] running on GCE node
I20260808 03:51:01.774173 27758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:01.775074 27758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:01.776305 27758 hybrid_clock.cc:648] HybridClock initialized: now 1786161061776213 us; error 50 us; skew 500 ppm
I20260808 03:51:01.776644 27758 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:01 dist-test-slave-jlzn krb5kdc[27219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161061, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:01.779963 27758 init.cc:377] Logged in from keytab as kudu/127.26.146.131@KRBTEST.COM (short username kudu)
I20260808 03:51:01.781517 27758 webserver.cc:460] Webserver started at http://127.26.146.131:39333/ using document root <none> and password file <none>
I20260808 03:51:01.782151 27758 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:01.782253 27758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:01.782511 27758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:01.785486 27758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "053b7fc59a484060b7a54669a119f8d4"
format_stamp: "Formatted at 2026-08-08 03:51:01 on dist-test-slave-jlzn"
server_key: "9a0c88fb78090f64c3cd368ee5ea2f94"
server_key_iv: "b1b5a95707035966d4ca2dc1bbcbca72"
server_key_version: "encryptionkey@0"
I20260808 03:51:01.786294 27758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "053b7fc59a484060b7a54669a119f8d4"
format_stamp: "Formatted at 2026-08-08 03:51:01 on dist-test-slave-jlzn"
server_key: "9a0c88fb78090f64c3cd368ee5ea2f94"
server_key_iv: "b1b5a95707035966d4ca2dc1bbcbca72"
server_key_version: "encryptionkey@0"
I20260808 03:51:01.791939 27758 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 03:51:01.795143 27776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:01.796480 27758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:51:01.796657 27758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "053b7fc59a484060b7a54669a119f8d4"
format_stamp: "Formatted at 2026-08-08 03:51:01 on dist-test-slave-jlzn"
server_key: "9a0c88fb78090f64c3cd368ee5ea2f94"
server_key_iv: "b1b5a95707035966d4ca2dc1bbcbca72"
server_key_version: "encryptionkey@0"
I20260808 03:51:01.796837 27758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:01.818604 27758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:01.822674 27758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:01.822999 27758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:01.824182 27758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:01.825637 27758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:01.825783 27758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:01.825911 27758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:01.825968 27758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:01.862473 27758 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:39049
I20260808 03:51:01.862619 27932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:39049 every 8 connection(s)
I20260808 03:51:01.863857 27758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:51:01.868144 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 27758
I20260808 03:51:01.868345 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:51:01.868779 27210 external_mini_cluster.cc:1428] Setting key b026a2d15223254ee9e71ca4cfc005be
Aug 08 03:51:01 dist-test-slave-jlzn krb5kdc[27219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161061, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:01.884801 27937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:01.866512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:53165 (local address 127.26.146.190:45413)
0808 03:51:01.867069 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:01.867074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:01.868150 (+ 1076us) server_negotiation.cc:407] Connection header received
0808 03:51:01.870587 (+ 2437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:01.870592 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:01.870671 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:01.870938 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:01.872662 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:01.873430 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:01.874922 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:01.875330 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:01.875434 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:01.878958 (+ 3524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:01.878963 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:01.878966 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:01.878998 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:01.881587 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:01.882358 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:01.882363 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:01.882365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:01.882485 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:01.882943 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:01.882947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:01.882948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:01.883227 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:01.883378 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:01.884187 (+ 809us) server_negotiation.cc:299] Negotiation successful
0808 03:51:01.884417 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":320,"threads_started":1}
I20260808 03:51:01.885659 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:01.866967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:45413 (local address 127.26.146.131:53165)
0808 03:51:01.867870 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:01.867918 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:01.870336 (+ 2418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:01.871190 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:01.871197 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:01.871746 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:01.872495 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:01.872523 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:01.873582 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:01.873586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:01.874789 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:01.874801 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:01.875550 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:01.875558 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:01.875764 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:01.876445 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:01.876471 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:01.878757 (+ 2286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:01.881731 (+ 2974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:01.881740 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:01.881756 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:01.882236 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:01.882588 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:01.882593 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:01.882596 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:01.882825 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:01.883352 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:01.883359 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:01.883736 (+ 377us) client_negotiation.cc:769] Sending connection context
0808 03:51:01.884046 (+ 310us) client_negotiation.cc:240] Negotiation successful
0808 03:51:01.884547 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":391,"threads_started":1}
I20260808 03:51:01.886866 27934 heartbeater.cc:347] Connected to a master server at 127.26.146.190:45413
I20260808 03:51:01.887147 27934 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:01.887882 27934 heartbeater.cc:511] Master 127.26.146.190:45413 requested a full tablet report, sending...
I20260808 03:51:01.889202 27275 ts_manager.cc:194] Registered new tserver with Master: 053b7fc59a484060b7a54669a119f8d4 (127.26.146.131:39049)
I20260808 03:51:01.890008 27275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.131@KRBTEST.COM'} at 127.26.146.131:53165
I20260808 03:51:01.896993 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 03:51:01 dist-test-slave-jlzn krb5kdc[27219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161061, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 03:51:01 dist-test-slave-jlzn krb5kdc[27219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161061, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.146.129@KRBTEST.COM
I20260808 03:51:01.949574 27950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:01.933847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37722 (local address 127.26.146.129:43261)
0808 03:51:01.934557 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:01.934566 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:01.934603 (+ 37us) server_negotiation.cc:407] Connection header received
0808 03:51:01.934692 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:01.934701 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:01.934964 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:01.935097 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:01.936843 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:01.937950 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:01.939394 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:01.940255 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:01.940351 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:01.944292 (+ 3941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:01.944303 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:01.944318 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:01.944348 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:01.947228 (+ 2880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:01.947843 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:01.947850 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:01.947852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:01.947974 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:01.948287 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:01.948289 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:01.948291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:01.948605 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:01.948834 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:01.949020 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 03:51:01.949223 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":335,"threads_started":1}
W20260808 03:51:01.950861 27426 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37722
Aug 08 03:51:01 dist-test-slave-jlzn krb5kdc[27219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 03:51:01 dist-test-slave-jlzn krb5kdc[27219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.129@KRBTEST.COM
I20260808 03:51:01.998512 27950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:01.984058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37724 (local address 127.26.146.129:43261)
0808 03:51:01.984312 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:01.984318 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:01.984493 (+ 175us) server_negotiation.cc:407] Connection header received
0808 03:51:01.984638 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:01.984642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:01.984708 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:01.984790 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:01.986043 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:01.986713 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:01.988084 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:01.988532 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:01.988622 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:01.992398 (+ 3776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:01.992403 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:01.992406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:01.992438 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:01.995701 (+ 3263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:01.996458 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:01.996464 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:01.996468 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:01.996626 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:01.997012 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:01.997018 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:01.997022 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:01.997360 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:01.997625 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:01.997939 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 03:51:01.998178 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 03:51:02.000603 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 27368
I20260808 03:51:02.017321 27950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:02.005504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37732 (local address 127.26.146.129:43261)
0808 03:51:02.005680 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:02.005685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:02.005826 (+ 141us) server_negotiation.cc:407] Connection header received
0808 03:51:02.005965 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:02.005970 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:02.006053 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:02.006143 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:02.007599 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:02.008484 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:02.009748 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:02.010167 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:02.010282 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:02.012537 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:02.012542 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:02.012545 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:02.012574 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:02.014997 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:02.015587 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:02.015593 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:02.015596 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:02.015774 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:02.016137 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:02.016140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:02.016142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:02.016465 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:02.016712 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:02.016898 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 03:51:02.017047 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 03:51:02.018013 27551 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:02.101137 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 27368
I20260808 03:51:02.122834 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 27564
Aug 08 03:51:02 dist-test-slave-jlzn krb5kdc[27219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.130@KRBTEST.COM
I20260808 03:51:02.137995 27962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:02.123779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37254 (local address 127.26.146.130:39201)
0808 03:51:02.124332 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:02.124339 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:02.124365 (+ 26us) server_negotiation.cc:407] Connection header received
0808 03:51:02.124429 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:02.124436 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:02.124660 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:02.124787 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:02.125872 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:02.126879 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:02.128172 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:02.128719 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:02.128813 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:02.132640 (+ 3827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:02.132652 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:02.132666 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:02.132700 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:02.135477 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:02.136259 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:02.136266 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:02.136268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:02.136390 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:02.136711 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:02.136714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:02.136716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:02.137034 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:02.137262 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:02.137460 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 03:51:02.137678 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":273,"threads_started":1}
I20260808 03:51:02.139243 27738 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:02.219842 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 27564
I20260808 03:51:02.241255 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 27758
Aug 08 03:51:02 dist-test-slave-jlzn krb5kdc[27219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.131@KRBTEST.COM
I20260808 03:51:02.257504 27964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:02.242273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38810 (local address 127.26.146.131:39049)
0808 03:51:02.242913 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:02.242920 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:02.242948 (+ 28us) server_negotiation.cc:407] Connection header received
0808 03:51:02.243008 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:02.243014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:02.243248 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:02.243372 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:02.244799 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:02.245835 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:02.247215 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:02.247757 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:02.247893 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:02.251893 (+ 4000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:02.251904 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:02.251920 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:02.251959 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:02.254672 (+ 2713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:02.255351 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:02.255358 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:02.255360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:02.255487 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:02.255993 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:02.255996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:02.255998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:02.256349 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:02.256585 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:02.256926 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 03:51:02.257159 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":331,"threads_started":1}
I20260808 03:51:02.258798 27925 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:02.341213 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 27758
I20260808 03:51:02.363240 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 27235
I20260808 03:51:02.364532 27324 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:02.467016 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 27235
2026-08-08T03:51:02Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4324 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:51:02 dist-test-slave-jlzn krb5kdc[27970](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:51:02 dist-test-slave-jlzn krb5kdc[27970](info): set up 2 sockets
Aug 08 03:51:02 dist-test-slave-jlzn krb5kdc[27970](info): commencing operation
krb5kdc: starting...
W20260808 03:51:03.641165 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:51:03 dist-test-slave-jlzn krb5kdc[27970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:51:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:51:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:51:03.827239 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:41329
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:41329
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:04.142805 27986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:04.143088 27986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:04.143225 27986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:04.150343 27986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:51:04.150449 27986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:04.150519 27986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:04.150595 27986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:51:04.150658 27986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:51:04.160441 27986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:41329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:41329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:04.161731 27986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:04.163041 27986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:04.174244 27995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:04.174269 27997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:04.174269 28000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:04.175251 27986 server_base.cc:1034] running on GCE node
I20260808 03:51:04.176056 27986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:04.177380 27986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:04.178576 27986 hybrid_clock.cc:648] HybridClock initialized: now 1786161064178529 us; error 37 us; skew 500 ppm
I20260808 03:51:04.178920 27986 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:04 dist-test-slave-jlzn krb5kdc[27970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161064, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:04.184917 27986 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:51:04.186451 27986 webserver.cc:460] Webserver started at http://127.26.146.190:41571/ using document root <none> and password file <none>
I20260808 03:51:04.187130 27986 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:04.187247 27986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:04.187556 27986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:04.190567 27986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "5ce27693510c4672bbef79a2324ffd81"
format_stamp: "Formatted at 2026-08-08 03:51:04 on dist-test-slave-jlzn"
server_key: "2aadaeacbab426ca85544e284aba598c"
server_key_iv: "71599ef5ec5cab9b71623e4e71022037"
server_key_version: "encryptionkey@0"
I20260808 03:51:04.191310 27986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "5ce27693510c4672bbef79a2324ffd81"
format_stamp: "Formatted at 2026-08-08 03:51:04 on dist-test-slave-jlzn"
server_key: "2aadaeacbab426ca85544e284aba598c"
server_key_iv: "71599ef5ec5cab9b71623e4e71022037"
server_key_version: "encryptionkey@0"
I20260808 03:51:04.196383 27986 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260808 03:51:04.199694 28010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:04.200918 27986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:51:04.201071 27986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "5ce27693510c4672bbef79a2324ffd81"
format_stamp: "Formatted at 2026-08-08 03:51:04 on dist-test-slave-jlzn"
server_key: "2aadaeacbab426ca85544e284aba598c"
server_key_iv: "71599ef5ec5cab9b71623e4e71022037"
server_key_version: "encryptionkey@0"
I20260808 03:51:04.201242 27986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:04.220656 27986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:04.224779 27986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:04.225046 27986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:04.242072 27986 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:41329
I20260808 03:51:04.242098 28095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:41329 every 8 connection(s)
I20260808 03:51:04.243427 27986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:51:04.247085 28097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:04.252179 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 27986
I20260808 03:51:04.252426 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:51:04.253225 27210 external_mini_cluster.cc:1428] Setting key 00878486909e0ce0af7e6402609073a6
I20260808 03:51:04.258646 28097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ce27693510c4672bbef79a2324ffd81: Bootstrap starting.
I20260808 03:51:04.262220 28097 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ce27693510c4672bbef79a2324ffd81: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:04.263443 28097 log.cc:824] T 00000000000000000000000000000000 P 5ce27693510c4672bbef79a2324ffd81: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:04.266256 28097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ce27693510c4672bbef79a2324ffd81: No bootstrap required, opened a new log
Aug 08 03:51:04 dist-test-slave-jlzn krb5kdc[27970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:04.275424 28097 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ce27693510c4672bbef79a2324ffd81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ce27693510c4672bbef79a2324ffd81" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 41329 } }
I20260808 03:51:04.275839 28097 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ce27693510c4672bbef79a2324ffd81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:04.275936 28102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:04.255260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48500 (local address 127.26.146.190:41329)
0808 03:51:04.256068 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:04.256081 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:04.256127 (+ 46us) server_negotiation.cc:407] Connection header received
0808 03:51:04.257829 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:04.257853 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:04.258217 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:04.258525 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:04.260344 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:04.261613 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:04.263267 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:04.264095 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:04.264307 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:04.268488 (+ 4181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:04.268496 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:04.268511 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:04.268556 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:04.271726 (+ 3170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:04.272567 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:04.272574 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:04.272580 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:04.272764 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:04.273141 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:04.273144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:04.273146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:04.273711 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:04.273956 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:04.274479 (+ 523us) server_negotiation.cc:299] Negotiation successful
0808 03:51:04.274851 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":354,"threads_started":1}
I20260808 03:51:04.275951 28097 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ce27693510c4672bbef79a2324ffd81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce27693510c4672bbef79a2324ffd81, State: Initialized, Role: FOLLOWER
I20260808 03:51:04.276605 28097 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ce27693510c4672bbef79a2324ffd81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ce27693510c4672bbef79a2324ffd81" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 41329 } }
I20260808 03:51:04.276846 28097 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ce27693510c4672bbef79a2324ffd81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:51:04.276976 28097 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ce27693510c4672bbef79a2324ffd81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:51:04.277145 28097 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ce27693510c4672bbef79a2324ffd81 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:04.280023 28097 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ce27693510c4672bbef79a2324ffd81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ce27693510c4672bbef79a2324ffd81" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 41329 } }
I20260808 03:51:04.280457 28097 leader_election.cc:302] T 00000000000000000000000000000000 P 5ce27693510c4672bbef79a2324ffd81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ce27693510c4672bbef79a2324ffd81; no voters:
I20260808 03:51:04.280985 28097 leader_election.cc:288] T 00000000000000000000000000000000 P 5ce27693510c4672bbef79a2324ffd81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:51:04.281104 28104 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ce27693510c4672bbef79a2324ffd81 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:04.281715 28104 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ce27693510c4672bbef79a2324ffd81 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce27693510c4672bbef79a2324ffd81, State: Running, Role: LEADER
I20260808 03:51:04.282270 28104 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ce27693510c4672bbef79a2324ffd81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ce27693510c4672bbef79a2324ffd81" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 41329 } }
I20260808 03:51:04.282863 28097 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ce27693510c4672bbef79a2324ffd81 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:51:04.285588 28106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ce27693510c4672bbef79a2324ffd81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ce27693510c4672bbef79a2324ffd81. Latest consensus state: current_term: 1 leader_uuid: "5ce27693510c4672bbef79a2324ffd81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ce27693510c4672bbef79a2324ffd81" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 41329 } } }
I20260808 03:51:04.285470 28105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ce27693510c4672bbef79a2324ffd81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ce27693510c4672bbef79a2324ffd81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ce27693510c4672bbef79a2324ffd81" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 41329 } } }
I20260808 03:51:04.285975 28106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ce27693510c4672bbef79a2324ffd81 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:04.285980 28105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ce27693510c4672bbef79a2324ffd81 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:04.287636 28118 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:51:04.292714 28118 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:51:04.301147 28118 catalog_manager.cc:1347] Generated new cluster ID: 637749a537a546628c238b7bba9a7347
I20260808 03:51:04.301255 28118 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:51:04.320415 28118 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:51:04.321753 28118 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:51:04.333248 28118 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ce27693510c4672bbef79a2324ffd81: Generated new TSK 0
I20260808 03:51:04.334090 28118 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
I20260808 03:51:04.387662 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:41329
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:04.706369 28139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:04.706696 28139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:04.706854 28139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:04.713969 28139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:04.714071 28139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:04.714227 28139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:51:04.723644 28139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:41329
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:04.724931 28139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:04.726226 28139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:04.738976 28151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:04.738901 28153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:04.739199 28156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:04.739583 28139 server_base.cc:1034] running on GCE node
I20260808 03:51:04.740213 28139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:04.741081 28139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:04.742306 28139 hybrid_clock.cc:648] HybridClock initialized: now 1786161064742246 us; error 51 us; skew 500 ppm
I20260808 03:51:04.742595 28139 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:04 dist-test-slave-jlzn krb5kdc[27970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161064, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:04.745649 28139 init.cc:377] Logged in from keytab as kudu/127.26.146.129@KRBTEST.COM (short username kudu)
I20260808 03:51:04.746968 28139 webserver.cc:460] Webserver started at http://127.26.146.129:35011/ using document root <none> and password file <none>
I20260808 03:51:04.747611 28139 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:04.747736 28139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:04.748054 28139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:04.750837 28139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "d7195421d9574d1484453d0becb93005"
format_stamp: "Formatted at 2026-08-08 03:51:04 on dist-test-slave-jlzn"
server_key: "44f94bbd60cad27640b5245757799c64"
server_key_iv: "660b63eecb0c5ff8687a97e649e40945"
server_key_version: "encryptionkey@0"
I20260808 03:51:04.751623 28139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "d7195421d9574d1484453d0becb93005"
format_stamp: "Formatted at 2026-08-08 03:51:04 on dist-test-slave-jlzn"
server_key: "44f94bbd60cad27640b5245757799c64"
server_key_iv: "660b63eecb0c5ff8687a97e649e40945"
server_key_version: "encryptionkey@0"
I20260808 03:51:04.756491 28139 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 03:51:04.759727 28168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:04.760952 28139 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 03:51:04.761098 28139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "d7195421d9574d1484453d0becb93005"
format_stamp: "Formatted at 2026-08-08 03:51:04 on dist-test-slave-jlzn"
server_key: "44f94bbd60cad27640b5245757799c64"
server_key_iv: "660b63eecb0c5ff8687a97e649e40945"
server_key_version: "encryptionkey@0"
I20260808 03:51:04.761291 28139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:04.781157 28139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:04.785076 28139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:04.785334 28139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:04.786260 28139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:04.787544 28139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:04.787624 28139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:04.787746 28139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:04.787817 28139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:04.817282 28139 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:33343
I20260808 03:51:04.817327 28328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:33343 every 8 connection(s)
I20260808 03:51:04.818542 28139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:51:04.826347 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 28139
I20260808 03:51:04.826545 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:51:04.827301 27210 external_mini_cluster.cc:1428] Setting key 6ed361974ae0f85c6a9f0e7d7d53b64e
Aug 08 03:51:04 dist-test-slave-jlzn krb5kdc[27970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161064, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:04.838954 28332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:04.821046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:40661 (local address 127.26.146.190:41329)
0808 03:51:04.821537 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:04.821543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:04.822483 (+ 940us) server_negotiation.cc:407] Connection header received
0808 03:51:04.824279 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:04.824284 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:04.824353 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:04.824449 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:04.825839 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:04.826648 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:04.828712 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:04.829296 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:04.829419 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:04.833308 (+ 3889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:04.833313 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:04.833316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:04.833349 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:04.835998 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:04.836693 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:04.836698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:04.836700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:04.836819 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:04.837237 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:04.837241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:04.837243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:04.837522 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:04.837651 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:04.838354 (+ 703us) server_negotiation.cc:299] Negotiation successful
0808 03:51:04.838601 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":238,"threads_started":1}
I20260808 03:51:04.839946 28333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:04.821633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:41329 (local address 127.26.146.129:40661)
0808 03:51:04.822324 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:04.822364 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:04.824022 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:04.824633 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:04.824642 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:04.825123 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:04.825689 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:04.825707 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:04.826819 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:04.826825 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:04.828561 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:04.828574 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:04.829529 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:04.829537 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:04.829816 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:04.830487 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:04.830519 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:04.833092 (+ 2573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:04.836134 (+ 3042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:04.836143 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:04.836159 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:04.836573 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:04.836922 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:04.836927 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:04.836931 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:04.837123 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:04.837645 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:04.837653 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:04.837977 (+ 324us) client_negotiation.cc:769] Sending connection context
0808 03:51:04.838233 (+ 256us) client_negotiation.cc:240] Negotiation successful
0808 03:51:04.838745 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":259,"threads_started":1}
I20260808 03:51:04.841559 28329 heartbeater.cc:347] Connected to a master server at 127.26.146.190:41329
I20260808 03:51:04.841848 28329 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:04.842572 28329 heartbeater.cc:511] Master 127.26.146.190:41329 requested a full tablet report, sending...
I20260808 03:51:04.844812 28039 ts_manager.cc:194] Registered new tserver with Master: d7195421d9574d1484453d0becb93005 (127.26.146.129:33343)
I20260808 03:51:04.846266 28039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.129@KRBTEST.COM'} at 127.26.146.129:40661
WARNING: no policy specified for kudu/127.26.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
I20260808 03:51:04.872201 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:41329
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:05.190207 28337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:05.190474 28337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:05.190604 28337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:05.197593 28337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:05.197698 28337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:05.197844 28337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:51:05.207393 28337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:41329
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:05.208774 28337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:05.210057 28337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:05.222534 28350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:05.222599 28345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:05.222599 28346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:05.223305 28337 server_base.cc:1034] running on GCE node
I20260808 03:51:05.223950 28337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:05.224862 28337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:05.226044 28337 hybrid_clock.cc:648] HybridClock initialized: now 1786161065225986 us; error 50 us; skew 500 ppm
I20260808 03:51:05.226337 28337 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:05 dist-test-slave-jlzn krb5kdc[27970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161065, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:05.229525 28337 init.cc:377] Logged in from keytab as kudu/127.26.146.130@KRBTEST.COM (short username kudu)
I20260808 03:51:05.230858 28337 webserver.cc:460] Webserver started at http://127.26.146.130:44495/ using document root <none> and password file <none>
I20260808 03:51:05.231501 28337 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:05.231629 28337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:05.231992 28337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:05.234843 28337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "7a1271b1e6b047719e6959b1a4325021"
format_stamp: "Formatted at 2026-08-08 03:51:05 on dist-test-slave-jlzn"
server_key: "59cef49f42e08dc36a62f408eae5a0be"
server_key_iv: "89aa42fc4ee4e7b42bb9d5f85d0c18a6"
server_key_version: "encryptionkey@0"
I20260808 03:51:05.235607 28337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "7a1271b1e6b047719e6959b1a4325021"
format_stamp: "Formatted at 2026-08-08 03:51:05 on dist-test-slave-jlzn"
server_key: "59cef49f42e08dc36a62f408eae5a0be"
server_key_iv: "89aa42fc4ee4e7b42bb9d5f85d0c18a6"
server_key_version: "encryptionkey@0"
I20260808 03:51:05.240763 28337 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 03:51:05.244160 28357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:05.245455 28337 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 03:51:05.245604 28337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "7a1271b1e6b047719e6959b1a4325021"
format_stamp: "Formatted at 2026-08-08 03:51:05 on dist-test-slave-jlzn"
server_key: "59cef49f42e08dc36a62f408eae5a0be"
server_key_iv: "89aa42fc4ee4e7b42bb9d5f85d0c18a6"
server_key_version: "encryptionkey@0"
I20260808 03:51:05.245788 28337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:05.270756 28337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:05.274966 28337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:05.275236 28337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:05.276206 28337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:05.277487 28337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:05.277566 28337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:05.277694 28337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:05.277748 28337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:05.307133 28337 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:46677
I20260808 03:51:05.307173 28523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:46677 every 8 connection(s)
I20260808 03:51:05.308398 28337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:51:05.310048 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 28337
I20260808 03:51:05.310235 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:51:05.310678 27210 external_mini_cluster.cc:1428] Setting key 73e4deb568caa7e94048de22c0cf8a94
Aug 08 03:51:05 dist-test-slave-jlzn krb5kdc[27970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161065, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:05.329641 28332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:05.310883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:55847 (local address 127.26.146.190:41329)
0808 03:51:05.311101 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:05.311107 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:05.312383 (+ 1276us) server_negotiation.cc:407] Connection header received
0808 03:51:05.314773 (+ 2390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:05.314777 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:05.314853 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:05.314934 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:05.316880 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:05.317598 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:05.319564 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:05.320025 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:05.320144 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:05.323303 (+ 3159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:05.323309 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:05.323313 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:05.323349 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:05.326532 (+ 3183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:05.327314 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:05.327320 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:05.327323 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:05.327495 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:05.327920 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:05.327925 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:05.327929 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:05.328278 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:05.328432 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:05.329060 (+ 628us) server_negotiation.cc:299] Negotiation successful
0808 03:51:05.329289 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 03:51:05.330324 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:05.311446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:41329 (local address 127.26.146.130:55847)
0808 03:51:05.312231 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:05.312270 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:05.314467 (+ 2197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:05.315203 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:05.315213 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:05.315914 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:05.316705 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:05.316728 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:05.317751 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:05.317756 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:05.319451 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:05.319465 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:05.320228 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:05.320233 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:05.320420 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:05.321037 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:05.321066 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:05.323097 (+ 2031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:05.326697 (+ 3600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:05.326706 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:05.326722 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:05.327179 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:05.327593 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:05.327596 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:05.327598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:05.327788 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:05.328395 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:05.328400 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:05.328687 (+ 287us) client_negotiation.cc:769] Sending connection context
0808 03:51:05.328908 (+ 221us) client_negotiation.cc:240] Negotiation successful
0808 03:51:05.329290 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":319,"threads_started":1}
WARNING: no policy specified for kudu/127.26.146.131@KRBTEST.COM; defaulting to no policy
I20260808 03:51:05.331629 28524 heartbeater.cc:347] Connected to a master server at 127.26.146.190:41329
I20260808 03:51:05.331956 28524 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:05.332723 28524 heartbeater.cc:511] Master 127.26.146.190:41329 requested a full tablet report, sending...
I20260808 03:51:05.334206 28039 ts_manager.cc:194] Registered new tserver with Master: 7a1271b1e6b047719e6959b1a4325021 (127.26.146.130:46677)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.131@KRBTEST.COM" created.
I20260808 03:51:05.335006 28039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.130@KRBTEST.COM'} at 127.26.146.130:55847
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
I20260808 03:51:05.356561 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:41329
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:05.715955 28531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:05.716228 28531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:05.716331 28531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:05.724182 28531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:05.724298 28531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:05.724421 28531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:51:05.734750 28531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:41329
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:05.736106 28531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:05.737401 28531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:05.750840 28541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:05.750811 28549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:05.750874 28542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:05.752043 28531 server_base.cc:1034] running on GCE node
I20260808 03:51:05.752863 28531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:05.753791 28531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:05.754978 28531 hybrid_clock.cc:648] HybridClock initialized: now 1786161065754932 us; error 39 us; skew 500 ppm
I20260808 03:51:05.755263 28531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:05 dist-test-slave-jlzn krb5kdc[27970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161065, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:05.758610 28531 init.cc:377] Logged in from keytab as kudu/127.26.146.131@KRBTEST.COM (short username kudu)
I20260808 03:51:05.759968 28531 webserver.cc:460] Webserver started at http://127.26.146.131:38383/ using document root <none> and password file <none>
I20260808 03:51:05.760579 28531 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:05.760689 28531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:05.760949 28531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:05.764047 28531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "03c8f68cf73b4e868e5f88f7b2c8ca41"
format_stamp: "Formatted at 2026-08-08 03:51:05 on dist-test-slave-jlzn"
server_key: "711ec70f861587514e3a7ec2f38c8e78"
server_key_iv: "4771fab93d817aff835273aaa8645d02"
server_key_version: "encryptionkey@0"
I20260808 03:51:05.764894 28531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "03c8f68cf73b4e868e5f88f7b2c8ca41"
format_stamp: "Formatted at 2026-08-08 03:51:05 on dist-test-slave-jlzn"
server_key: "711ec70f861587514e3a7ec2f38c8e78"
server_key_iv: "4771fab93d817aff835273aaa8645d02"
server_key_version: "encryptionkey@0"
I20260808 03:51:05.769932 28531 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:51:05.773476 28560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:05.774744 28531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 03:51:05.774899 28531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "03c8f68cf73b4e868e5f88f7b2c8ca41"
format_stamp: "Formatted at 2026-08-08 03:51:05 on dist-test-slave-jlzn"
server_key: "711ec70f861587514e3a7ec2f38c8e78"
server_key_iv: "4771fab93d817aff835273aaa8645d02"
server_key_version: "encryptionkey@0"
I20260808 03:51:05.775090 28531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:05.793820 28531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:05.797945 28531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:05.798204 28531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:05.799938 28531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:05.801273 28531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:05.801347 28531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:05.801440 28531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:05.801496 28531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:05.849213 28329 heartbeater.cc:503] Master 127.26.146.190:41329 was elected leader, sending a full tablet report...
I20260808 03:51:05.861688 28531 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:36079
I20260808 03:51:05.861840 28723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:36079 every 8 connection(s)
I20260808 03:51:05.863370 28531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:51:05.864298 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 28531
I20260808 03:51:05.864527 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:51:05.865139 27210 external_mini_cluster.cc:1428] Setting key 5b34ed25ac3fad7b641054e8d9a6a452
Aug 08 03:51:05 dist-test-slave-jlzn krb5kdc[27970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161065, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:05.889701 28726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:05.866945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:46913 (local address 127.26.146.190:41329)
0808 03:51:05.867848 (+ 903us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:05.867853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:05.869572 (+ 1719us) server_negotiation.cc:407] Connection header received
0808 03:51:05.872180 (+ 2608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:05.872184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:05.872250 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:05.872341 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:05.874263 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:05.875276 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:05.877346 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:05.878014 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:05.878139 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:05.882954 (+ 4815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:05.882961 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:05.882966 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:05.883007 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:05.886383 (+ 3376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:05.887088 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:05.887092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:05.887094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:05.887275 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:05.887744 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:05.887747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:05.887749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:05.888148 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:05.888261 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:05.889031 (+ 770us) server_negotiation.cc:299] Negotiation successful
0808 03:51:05.889238 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":523,"threads_started":1}
I20260808 03:51:05.890763 28727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:05.867969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:41329 (local address 127.26.146.131:46913)
0808 03:51:05.869415 (+ 1446us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:05.869454 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:05.871938 (+ 2484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:05.872654 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:05.872662 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:05.873322 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:05.874104 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:05.874124 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:05.875479 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:05.875483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:05.877199 (+ 1716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:05.877212 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:05.878342 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:05.878353 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:05.878653 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:05.879358 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:05.879416 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:05.882722 (+ 3306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:05.886502 (+ 3780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:05.886509 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:05.886521 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:05.886996 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:05.887358 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:05.887361 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:05.887363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:05.887653 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:05.888275 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:05.888282 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:05.888707 (+ 425us) client_negotiation.cc:769] Sending connection context
0808 03:51:05.888969 (+ 262us) client_negotiation.cc:240] Negotiation successful
0808 03:51:05.889390 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1179,"thread_start_us":917,"threads_started":1}
I20260808 03:51:05.892365 28724 heartbeater.cc:347] Connected to a master server at 127.26.146.190:41329
I20260808 03:51:05.892706 28724 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:05.894009 28724 heartbeater.cc:511] Master 127.26.146.190:41329 requested a full tablet report, sending...
I20260808 03:51:05.895615 28039 ts_manager.cc:194] Registered new tserver with Master: 03c8f68cf73b4e868e5f88f7b2c8ca41 (127.26.146.131:36079)
I20260808 03:51:05.896663 28039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.131@KRBTEST.COM'} at 127.26.146.131:46913
I20260808 03:51:05.906144 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:51:05.930892 28726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:05.914464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48502 (local address 127.26.146.190:41329)
0808 03:51:05.914629 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:05.914634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:05.915549 (+ 915us) server_negotiation.cc:407] Connection header received
0808 03:51:05.915627 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:05.915630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:05.915707 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:05.915780 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:05.918309 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:05.919297 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:05.921164 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:05.921687 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:05.921818 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:05.924619 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:05.924624 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:05.924627 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:05.924660 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:05.928280 (+ 3620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:05.928957 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:05.928960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:05.928962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:05.929136 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:05.929454 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:05.929457 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:05.929459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:05.929785 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:05.929989 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:05.930400 (+ 411us) server_negotiation.cc:299] Negotiation successful
0808 03:51:05.930595 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 03:51:05.937531 28039 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:51:05.949932 28039 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:51:05.993407 28745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:05.966086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39852 (local address 127.26.146.130:46677)
0808 03:51:05.980459 (+ 14373us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:05.980470 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:05.980504 (+ 34us) server_negotiation.cc:407] Connection header received
0808 03:51:05.980613 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:05.980620 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:05.980996 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:05.981235 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:05.983261 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:05.985782 (+ 2521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:05.989313 (+ 3531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:05.991619 (+ 2306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:05.991880 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:05.992076 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:05.992574 (+ 498us) server_negotiation.cc:299] Negotiation successful
0808 03:51:05.992850 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13281,"thread_start_us":13577,"threads_started":1}
I20260808 03:51:05.993420 28739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:05.966349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:46677 (local address 127.0.0.1:39852)
0808 03:51:05.979586 (+ 13237us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:05.979882 (+ 296us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:05.980059 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:05.981270 (+ 1211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:05.981279 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:05.981316 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:05.982862 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:05.982911 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:05.985945 (+ 3034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:05.985950 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:05.989117 (+ 3167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:05.989137 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:05.992031 (+ 2894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:05.992044 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:05.992349 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:05.992406 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 03:51:05.992510 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 03:51:05.992747 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13080,"thread_start_us":10963,"threads_started":1}
I20260808 03:51:05.998525 28425 tablet_service.cc:1467] Processing CreateTablet for tablet e7f32ce055ba47ad9eb2cd9072a5d544 (DEFAULT_TABLE table=test-table [id=12b7f4af38554dc6b99ade5a17d20269]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:06.000447 28425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7f32ce055ba47ad9eb2cd9072a5d544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:06.002645 28747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:05.966462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:36079 (local address 127.0.0.1:36178)
0808 03:51:05.989605 (+ 23143us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:05.989619 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:05.989770 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:05.990947 (+ 1177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:05.990952 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:05.990980 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:05.992067 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:05.992080 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:05.994794 (+ 2714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:05.994800 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:05.998814 (+ 4014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:05.998826 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:06.001064 (+ 2238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:06.001073 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:06.001407 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:06.001444 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 03:51:06.001521 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 03:51:06.001652 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23007,"mutex_wait_us":1,"thread_start_us":21443,"threads_started":1}
I20260808 03:51:06.002739 28748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:05.967767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36178 (local address 127.26.146.131:36079)
0808 03:51:05.987987 (+ 20220us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:05.987996 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:05.989717 (+ 1721us) server_negotiation.cc:407] Connection header received
0808 03:51:05.989950 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:05.989956 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:05.990099 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:05.990337 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:05.992449 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:05.994598 (+ 2149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:05.998956 (+ 4358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:06.000744 (+ 1788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:06.001017 (+ 273us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:06.001187 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:06.001641 (+ 454us) server_negotiation.cc:299] Negotiation successful
0808 03:51:06.001847 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17962,"thread_start_us":19223,"threads_started":1}
I20260808 03:51:06.003618 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:05.965656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:33343 (local address 127.0.0.1:35526)
0808 03:51:05.979114 (+ 13458us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:05.979168 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:05.979444 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:05.989783 (+ 10339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:05.989789 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:05.989819 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:05.990916 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:05.990933 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:05.994002 (+ 3069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:05.994007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:05.997937 (+ 3930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:05.997954 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:06.001251 (+ 3297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:06.001260 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:06.002528 (+ 1268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:06.002564 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 03:51:06.002661 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 03:51:06.002835 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11896,"thread_start_us":13360,"threads_started":1}
I20260808 03:51:06.005020 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:05.965876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35526 (local address 127.26.146.129:33343)
0808 03:51:05.989118 (+ 23242us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:05.989124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:05.989149 (+ 25us) server_negotiation.cc:407] Connection header received
0808 03:51:05.989231 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:05.989239 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:05.989524 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:05.991100 (+ 1576us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:05.991519 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:05.993867 (+ 2348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:05.998147 (+ 4280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:06.000454 (+ 2307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:06.000769 (+ 315us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:06.000951 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:06.004116 (+ 3165us) server_negotiation.cc:299] Negotiation successful
0808 03:51:06.004334 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23034,"thread_start_us":22740,"threads_started":1}
I20260808 03:51:06.007962 28633 tablet_service.cc:1467] Processing CreateTablet for tablet e7f32ce055ba47ad9eb2cd9072a5d544 (DEFAULT_TABLE table=test-table [id=12b7f4af38554dc6b99ade5a17d20269]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:06.009272 28239 tablet_service.cc:1467] Processing CreateTablet for tablet e7f32ce055ba47ad9eb2cd9072a5d544 (DEFAULT_TABLE table=test-table [id=12b7f4af38554dc6b99ade5a17d20269]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:06.010138 28633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7f32ce055ba47ad9eb2cd9072a5d544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:06.011006 28239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7f32ce055ba47ad9eb2cd9072a5d544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:06.025399 28749 tablet_bootstrap.cc:492] T e7f32ce055ba47ad9eb2cd9072a5d544 P 7a1271b1e6b047719e6959b1a4325021: Bootstrap starting.
I20260808 03:51:06.027542 28750 tablet_bootstrap.cc:492] T e7f32ce055ba47ad9eb2cd9072a5d544 P d7195421d9574d1484453d0becb93005: Bootstrap starting.
I20260808 03:51:06.031180 28753 tablet_bootstrap.cc:492] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41: Bootstrap starting.
I20260808 03:51:06.032837 28749 tablet_bootstrap.cc:654] T e7f32ce055ba47ad9eb2cd9072a5d544 P 7a1271b1e6b047719e6959b1a4325021: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:06.033218 28750 tablet_bootstrap.cc:654] T e7f32ce055ba47ad9eb2cd9072a5d544 P d7195421d9574d1484453d0becb93005: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:06.035187 28750 log.cc:824] T e7f32ce055ba47ad9eb2cd9072a5d544 P d7195421d9574d1484453d0becb93005: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:06.035476 28749 log.cc:824] T e7f32ce055ba47ad9eb2cd9072a5d544 P 7a1271b1e6b047719e6959b1a4325021: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:06.037787 28753 tablet_bootstrap.cc:654] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:06.039767 28749 tablet_bootstrap.cc:492] T e7f32ce055ba47ad9eb2cd9072a5d544 P 7a1271b1e6b047719e6959b1a4325021: No bootstrap required, opened a new log
I20260808 03:51:06.039832 28750 tablet_bootstrap.cc:492] T e7f32ce055ba47ad9eb2cd9072a5d544 P d7195421d9574d1484453d0becb93005: No bootstrap required, opened a new log
I20260808 03:51:06.039978 28753 log.cc:824] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:06.040210 28749 ts_tablet_manager.cc:1397] T e7f32ce055ba47ad9eb2cd9072a5d544 P 7a1271b1e6b047719e6959b1a4325021: Time spent bootstrapping tablet: real 0.016s user 0.013s sys 0.000s
I20260808 03:51:06.040318 28750 ts_tablet_manager.cc:1397] T e7f32ce055ba47ad9eb2cd9072a5d544 P d7195421d9574d1484453d0becb93005: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.012s
I20260808 03:51:06.045658 28753 tablet_bootstrap.cc:492] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41: No bootstrap required, opened a new log
I20260808 03:51:06.046155 28753 ts_tablet_manager.cc:1397] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.008s
I20260808 03:51:06.052626 28749 raft_consensus.cc:348] T e7f32ce055ba47ad9eb2cd9072a5d544 P 7a1271b1e6b047719e6959b1a4325021 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7195421d9574d1484453d0becb93005" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 33343 } } peers { permanent_uuid: "7a1271b1e6b047719e6959b1a4325021" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 46677 } } peers { permanent_uuid: "03c8f68cf73b4e868e5f88f7b2c8ca41" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36079 } }
I20260808 03:51:06.053020 28749 raft_consensus.cc:374] T e7f32ce055ba47ad9eb2cd9072a5d544 P 7a1271b1e6b047719e6959b1a4325021 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:06.053170 28749 raft_consensus.cc:729] T e7f32ce055ba47ad9eb2cd9072a5d544 P 7a1271b1e6b047719e6959b1a4325021 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a1271b1e6b047719e6959b1a4325021, State: Initialized, Role: FOLLOWER
I20260808 03:51:06.053741 28749 consensus_queue.cc:260] T e7f32ce055ba47ad9eb2cd9072a5d544 P 7a1271b1e6b047719e6959b1a4325021 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7195421d9574d1484453d0becb93005" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 33343 } } peers { permanent_uuid: "7a1271b1e6b047719e6959b1a4325021" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 46677 } } peers { permanent_uuid: "03c8f68cf73b4e868e5f88f7b2c8ca41" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36079 } }
I20260808 03:51:06.055473 28524 heartbeater.cc:503] Master 127.26.146.190:41329 was elected leader, sending a full tablet report...
I20260808 03:51:06.056146 28749 ts_tablet_manager.cc:1428] T e7f32ce055ba47ad9eb2cd9072a5d544 P 7a1271b1e6b047719e6959b1a4325021: Time spent starting tablet: real 0.016s user 0.009s sys 0.007s
I20260808 03:51:06.056092 28750 raft_consensus.cc:348] T e7f32ce055ba47ad9eb2cd9072a5d544 P d7195421d9574d1484453d0becb93005 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7195421d9574d1484453d0becb93005" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 33343 } } peers { permanent_uuid: "7a1271b1e6b047719e6959b1a4325021" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 46677 } } peers { permanent_uuid: "03c8f68cf73b4e868e5f88f7b2c8ca41" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36079 } }
I20260808 03:51:06.056536 28750 raft_consensus.cc:374] T e7f32ce055ba47ad9eb2cd9072a5d544 P d7195421d9574d1484453d0becb93005 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:06.056655 28750 raft_consensus.cc:729] T e7f32ce055ba47ad9eb2cd9072a5d544 P d7195421d9574d1484453d0becb93005 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7195421d9574d1484453d0becb93005, State: Initialized, Role: FOLLOWER
I20260808 03:51:06.057329 28750 consensus_queue.cc:260] T e7f32ce055ba47ad9eb2cd9072a5d544 P d7195421d9574d1484453d0becb93005 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7195421d9574d1484453d0becb93005" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 33343 } } peers { permanent_uuid: "7a1271b1e6b047719e6959b1a4325021" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 46677 } } peers { permanent_uuid: "03c8f68cf73b4e868e5f88f7b2c8ca41" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36079 } }
I20260808 03:51:06.059024 28750 ts_tablet_manager.cc:1428] T e7f32ce055ba47ad9eb2cd9072a5d544 P d7195421d9574d1484453d0becb93005: Time spent starting tablet: real 0.019s user 0.010s sys 0.007s
W20260808 03:51:06.060359 28525 tablet.cc:2369] T e7f32ce055ba47ad9eb2cd9072a5d544 P 7a1271b1e6b047719e6959b1a4325021: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:51:06.060351 28753 raft_consensus.cc:348] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7195421d9574d1484453d0becb93005" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 33343 } } peers { permanent_uuid: "7a1271b1e6b047719e6959b1a4325021" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 46677 } } peers { permanent_uuid: "03c8f68cf73b4e868e5f88f7b2c8ca41" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36079 } }
I20260808 03:51:06.060756 28753 raft_consensus.cc:374] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:06.060858 28753 raft_consensus.cc:729] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03c8f68cf73b4e868e5f88f7b2c8ca41, State: Initialized, Role: FOLLOWER
I20260808 03:51:06.061460 28753 consensus_queue.cc:260] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7195421d9574d1484453d0becb93005" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 33343 } } peers { permanent_uuid: "7a1271b1e6b047719e6959b1a4325021" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 46677 } } peers { permanent_uuid: "03c8f68cf73b4e868e5f88f7b2c8ca41" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36079 } }
I20260808 03:51:06.062657 28724 heartbeater.cc:503] Master 127.26.146.190:41329 was elected leader, sending a full tablet report...
I20260808 03:51:06.063239 28753 ts_tablet_manager.cc:1428] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41: Time spent starting tablet: real 0.017s user 0.014s sys 0.003s
W20260808 03:51:06.070710 28330 tablet.cc:2369] T e7f32ce055ba47ad9eb2cd9072a5d544 P d7195421d9574d1484453d0becb93005: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:51:06.116735 28725 tablet.cc:2369] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:51:06.200423 28765 raft_consensus.cc:482] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:51:06.200758 28765 raft_consensus.cc:504] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7195421d9574d1484453d0becb93005" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 33343 } } peers { permanent_uuid: "7a1271b1e6b047719e6959b1a4325021" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 46677 } } peers { permanent_uuid: "03c8f68cf73b4e868e5f88f7b2c8ca41" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36079 } }
I20260808 03:51:06.202888 28765 leader_election.cc:288] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7195421d9574d1484453d0becb93005 (127.26.146.129:33343), 7a1271b1e6b047719e6959b1a4325021 (127.26.146.130:46677)
I20260808 03:51:06.210112 28745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:06.203542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:58091 (local address 127.26.146.130:46677)
0808 03:51:06.203751 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:06.203756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:06.203777 (+ 21us) server_negotiation.cc:407] Connection header received
0808 03:51:06.203880 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:06.203885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:06.203956 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:06.204028 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:06.204462 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:06.205555 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:06.207898 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:06.209202 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:06.209353 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:06.209429 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:06.209728 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 03:51:06.209849 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 03:51:06.210172 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:06.203253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:49429 (local address 127.26.146.129:33343)
0808 03:51:06.203475 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:06.203481 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:06.204090 (+ 609us) server_negotiation.cc:407] Connection header received
0808 03:51:06.204224 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:06.204228 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:06.204287 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:06.204354 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:06.204864 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:06.205846 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:06.208103 (+ 2257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:06.209335 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:06.209465 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:06.209518 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:06.209843 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 03:51:06.209966 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 03:51:06.210196 28727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:06.203193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:46677 (local address 127.26.146.131:58091)
0808 03:51:06.203477 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:06.203496 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:06.203632 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:06.204076 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:06.204081 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:06.204159 (+ 78us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:06.204313 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:06.204321 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:06.205686 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:06.205690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:06.207685 (+ 1995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:06.207729 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:06.209347 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:06.209351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:06.209547 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:06.209592 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 03:51:06.209681 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 03:51:06.209827 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":20,"spinlock_wait_cycles":23936}
I20260808 03:51:06.210259 28766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:06.203249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:33343 (local address 127.26.146.131:49429)
0808 03:51:06.203977 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:06.203991 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:06.204106 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:06.204391 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:06.204396 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:06.204419 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:06.204708 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:06.204717 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:06.205985 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:06.205991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:06.207960 (+ 1969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:06.207971 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:06.209529 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:06.209535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:06.209701 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:06.209729 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 03:51:06.209802 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 03:51:06.209899 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"mutex_wait_us":6,"thread_start_us":409,"threads_started":1}
I20260808 03:51:06.211344 28269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7f32ce055ba47ad9eb2cd9072a5d544" candidate_uuid: "03c8f68cf73b4e868e5f88f7b2c8ca41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7195421d9574d1484453d0becb93005" is_pre_election: true
I20260808 03:51:06.211378 28458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7f32ce055ba47ad9eb2cd9072a5d544" candidate_uuid: "03c8f68cf73b4e868e5f88f7b2c8ca41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a1271b1e6b047719e6959b1a4325021" is_pre_election: true
I20260808 03:51:06.212102 28269 raft_consensus.cc:2454] T e7f32ce055ba47ad9eb2cd9072a5d544 P d7195421d9574d1484453d0becb93005 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03c8f68cf73b4e868e5f88f7b2c8ca41 in term 0.
I20260808 03:51:06.212779 28561 leader_election.cc:302] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03c8f68cf73b4e868e5f88f7b2c8ca41, d7195421d9574d1484453d0becb93005; no voters:
I20260808 03:51:06.213140 28765 raft_consensus.cc:2781] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:51:06.213286 28765 raft_consensus.cc:482] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:51:06.213385 28765 raft_consensus.cc:3037] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:06.213891 28458 raft_consensus.cc:2454] T e7f32ce055ba47ad9eb2cd9072a5d544 P 7a1271b1e6b047719e6959b1a4325021 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03c8f68cf73b4e868e5f88f7b2c8ca41 in term 0.
I20260808 03:51:06.216208 28765 raft_consensus.cc:504] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7195421d9574d1484453d0becb93005" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 33343 } } peers { permanent_uuid: "7a1271b1e6b047719e6959b1a4325021" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 46677 } } peers { permanent_uuid: "03c8f68cf73b4e868e5f88f7b2c8ca41" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36079 } }
I20260808 03:51:06.216791 28765 leader_election.cc:288] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41 [CANDIDATE]: Term 1 election: Requested vote from peers d7195421d9574d1484453d0becb93005 (127.26.146.129:33343), 7a1271b1e6b047719e6959b1a4325021 (127.26.146.130:46677)
I20260808 03:51:06.217141 28269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7f32ce055ba47ad9eb2cd9072a5d544" candidate_uuid: "03c8f68cf73b4e868e5f88f7b2c8ca41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7195421d9574d1484453d0becb93005"
I20260808 03:51:06.217275 28458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7f32ce055ba47ad9eb2cd9072a5d544" candidate_uuid: "03c8f68cf73b4e868e5f88f7b2c8ca41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a1271b1e6b047719e6959b1a4325021"
I20260808 03:51:06.217326 28269 raft_consensus.cc:3037] T e7f32ce055ba47ad9eb2cd9072a5d544 P d7195421d9574d1484453d0becb93005 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:06.217463 28458 raft_consensus.cc:3037] T e7f32ce055ba47ad9eb2cd9072a5d544 P 7a1271b1e6b047719e6959b1a4325021 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:06.219594 28458 raft_consensus.cc:2454] T e7f32ce055ba47ad9eb2cd9072a5d544 P 7a1271b1e6b047719e6959b1a4325021 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03c8f68cf73b4e868e5f88f7b2c8ca41 in term 1.
I20260808 03:51:06.219645 28269 raft_consensus.cc:2454] T e7f32ce055ba47ad9eb2cd9072a5d544 P d7195421d9574d1484453d0becb93005 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03c8f68cf73b4e868e5f88f7b2c8ca41 in term 1.
I20260808 03:51:06.219990 28565 leader_election.cc:302] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03c8f68cf73b4e868e5f88f7b2c8ca41, 7a1271b1e6b047719e6959b1a4325021; no voters:
I20260808 03:51:06.220273 28765 raft_consensus.cc:2781] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:06.220651 28765 raft_consensus.cc:686] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41 [term 1 LEADER]: Becoming Leader. State: Replica: 03c8f68cf73b4e868e5f88f7b2c8ca41, State: Running, Role: LEADER
I20260808 03:51:06.221160 28765 consensus_queue.cc:237] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7195421d9574d1484453d0becb93005" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 33343 } } peers { permanent_uuid: "7a1271b1e6b047719e6959b1a4325021" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 46677 } } peers { permanent_uuid: "03c8f68cf73b4e868e5f88f7b2c8ca41" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36079 } }
I20260808 03:51:06.225695 28038 catalog_manager.cc:5495] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 03c8f68cf73b4e868e5f88f7b2c8ca41 (127.26.146.131). New cstate: current_term: 1 leader_uuid: "03c8f68cf73b4e868e5f88f7b2c8ca41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7195421d9574d1484453d0becb93005" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 33343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a1271b1e6b047719e6959b1a4325021" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 46677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03c8f68cf73b4e868e5f88f7b2c8ca41" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36079 } health_report { overall_health: HEALTHY } } }
Aug 08 03:51:06 dist-test-slave-jlzn krb5kdc[27970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.129@KRBTEST.COM
I20260808 03:51:06.289371 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:06.274450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35538 (local address 127.26.146.129:33343)
0808 03:51:06.274678 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:06.274686 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:06.275129 (+ 443us) server_negotiation.cc:407] Connection header received
0808 03:51:06.275287 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:06.275291 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:06.275360 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:06.275446 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:06.276964 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:06.277885 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:06.279278 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:06.279740 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:06.279876 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:06.283795 (+ 3919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:06.283849 (+ 54us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:06.283882 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:06.283921 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:06.286893 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:06.287547 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:06.287554 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:06.287557 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:06.287699 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:06.288054 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:06.288058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:06.288060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:06.288447 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:06.288694 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:06.288873 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 03:51:06.289031 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
Aug 08 03:51:06 dist-test-slave-jlzn krb5kdc[27970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161066, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 03:51:06 dist-test-slave-jlzn krb5kdc[27970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161066, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.146.129@KRBTEST.COM
I20260808 03:51:06.351991 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:06.337937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35542 (local address 127.26.146.129:33343)
0808 03:51:06.338180 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:06.338187 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:06.338414 (+ 227us) server_negotiation.cc:407] Connection header received
0808 03:51:06.338562 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:06.338566 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:06.338646 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:06.338734 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:06.340095 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:06.340833 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:06.342637 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:06.343041 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:06.343134 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:06.346857 (+ 3723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:06.346862 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:06.346865 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:06.346898 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:06.349512 (+ 2614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:06.350085 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:06.350091 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:06.350094 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:06.350237 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:06.350603 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:06.350607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:06.350609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:06.350878 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:06.351113 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:06.351405 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 03:51:06.351640 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
Aug 08 03:51:06 dist-test-slave-jlzn krb5kdc[27970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260808 03:51:06.480644 27210 mini_kdc.cc:317] Time spent kinit for test-admin: real 0.126s user 0.000s sys 0.012s
Aug 08 03:51:06 dist-test-slave-jlzn krb5kdc[27970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.129@KRBTEST.COM
I20260808 03:51:06.496395 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:06.482807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35556 (local address 127.26.146.129:33343)
0808 03:51:06.483025 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:06.483031 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:06.483248 (+ 217us) server_negotiation.cc:407] Connection header received
0808 03:51:06.483446 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:06.483453 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:06.483526 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:06.483616 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:06.485068 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:06.485837 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:06.487089 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:06.487517 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:06.487626 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:06.491374 (+ 3748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:06.491379 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:06.491382 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:06.491416 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:06.493799 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:06.494458 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:06.494463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:06.494467 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:06.494594 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:06.494946 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:06.494952 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:06.494955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:06.495266 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:06.495499 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:06.495790 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 03:51:06.496051 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
W20260808 03:51:06.498560 28212 tablet_service.cc:2908] Rejecting scan request for tablet e7f32ce055ba47ad9eb2cd9072a5d544: Uninitialized: clean time has not yet been initialized
I20260808 03:51:06.502322 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 28139
I20260808 03:51:06.514442 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:06.503170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35570 (local address 127.26.146.129:33343)
0808 03:51:06.503379 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:06.503386 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:06.503877 (+ 491us) server_negotiation.cc:407] Connection header received
0808 03:51:06.504020 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:06.504025 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:06.504112 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:06.504232 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:06.505397 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:06.506110 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:06.507311 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:06.507716 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:06.507861 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:06.509957 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:06.509961 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:06.509964 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:06.509989 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:06.512283 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:06.512857 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:06.512861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:06.512863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:06.512964 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:06.513244 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:06.513247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:06.513249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:06.513502 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:06.513705 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:06.513946 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 03:51:06.514106 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 03:51:06.515098 28323 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:06.608716 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 28139
I20260808 03:51:06.629350 28765 consensus_queue.cc:1035] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a1271b1e6b047719e6959b1a4325021" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 46677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 03:51:06.632476 28561 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.146.129:33343 (error 108)
I20260808 03:51:06.633874 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 28337
I20260808 03:51:06.636572 28766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:06.636011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:33343 (local address 127.26.146.131:55761)
0808 03:51:06.636275 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:06.636404 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:33343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
W20260808 03:51:06.636873 28561 consensus_peers.cc:487] T e7f32ce055ba47ad9eb2cd9072a5d544 P 03c8f68cf73b4e868e5f88f7b2c8ca41 -> Peer d7195421d9574d1484453d0becb93005 (127.26.146.129:33343): Couldn't send request to peer d7195421d9574d1484453d0becb93005. Status: Network error: Client connection negotiation failed: client connection to 127.26.146.129:33343: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
Aug 08 03:51:06 dist-test-slave-jlzn krb5kdc[27970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.130@KRBTEST.COM
I20260808 03:51:06.649658 28745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:06.635028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39862 (local address 127.26.146.130:46677)
0808 03:51:06.635204 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:06.635210 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:06.635461 (+ 251us) server_negotiation.cc:407] Connection header received
0808 03:51:06.635736 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:06.635741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:06.635841 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:06.635911 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:06.637703 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:06.638657 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:06.640000 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:06.640388 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:06.640480 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:06.644150 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:06.644161 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:06.644174 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:06.644208 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:06.647198 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:06.647763 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:06.647769 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:06.647772 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:06.647940 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:06.648285 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:06.648289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:06.648291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:06.648671 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:06.648915 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:06.649178 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 03:51:06.649343 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 03:51:06.650269 28514 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:06.739094 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 28337
I20260808 03:51:06.763113 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 28531
Aug 08 03:51:06 dist-test-slave-jlzn krb5kdc[27970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.131@KRBTEST.COM
I20260808 03:51:06.777312 28804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:06.763962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36182 (local address 127.26.146.131:36079)
0808 03:51:06.764435 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:06.764439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:06.764456 (+ 17us) server_negotiation.cc:407] Connection header received
0808 03:51:06.764513 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:06.764517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:06.764577 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:06.764669 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:06.765828 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:06.766564 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:06.767781 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:06.768228 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:06.768318 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:06.772138 (+ 3820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:06.772153 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:06.772176 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:06.772225 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:06.774779 (+ 2554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:06.775374 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:06.775381 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:06.775383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:06.775517 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:06.775826 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:06.775831 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:06.775833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:06.776166 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:06.776408 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:06.776776 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 03:51:06.777003 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":257,"threads_started":1}
I20260808 03:51:06.777918 28712 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:06.866636 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 28531
I20260808 03:51:06.889199 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 27986
I20260808 03:51:06.890458 28088 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:06.998720 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 27986
2026-08-08T03:51:07Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4531 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:51:07 dist-test-slave-jlzn krb5kdc[28810](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:51:07 dist-test-slave-jlzn krb5kdc[28810](info): set up 2 sockets
Aug 08 03:51:07 dist-test-slave-jlzn krb5kdc[28810](info): commencing operation
krb5kdc: starting...
W20260808 03:51:08.515970 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.469s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:51:08 dist-test-slave-jlzn krb5kdc[28810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:51:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:51:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:51:08.706807 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:40575
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:40575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:09.027364 28829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:09.027647 28829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:09.027746 28829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:09.034942 28829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:51:09.035043 28829 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 03:51:09.035091 28829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:09.035159 28829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:09.035215 28829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:51:09.035275 28829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:51:09.044960 28829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:40575
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:40575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:09.046176 28829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:09.047390 28829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:09.058784 28834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:09.058753 28835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:09.058705 28837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:09.059602 28829 server_base.cc:1034] running on GCE node
I20260808 03:51:09.060364 28829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:09.061671 28829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:09.062889 28829 hybrid_clock.cc:648] HybridClock initialized: now 1786161069062826 us; error 33 us; skew 500 ppm
I20260808 03:51:09.063201 28829 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:09 dist-test-slave-jlzn krb5kdc[28810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161069, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:09.066223 28829 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:51:09.067562 28829 webserver.cc:460] Webserver started at http://127.26.146.190:34729/ using document root <none> and password file <none>
I20260808 03:51:09.068276 28829 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:09.068379 28829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:09.068701 28829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:09.071660 28829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "dda29ae6a2ca4edcbb41428c7a476a7f"
format_stamp: "Formatted at 2026-08-08 03:51:09 on dist-test-slave-jlzn"
server_key: "204d8bcc9a6b9d2469201ad8284aa6a7"
server_key_iv: "d83aee8b4df9557bc71735b73a17d514"
server_key_version: "encryptionkey@0"
I20260808 03:51:09.072510 28829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "dda29ae6a2ca4edcbb41428c7a476a7f"
format_stamp: "Formatted at 2026-08-08 03:51:09 on dist-test-slave-jlzn"
server_key: "204d8bcc9a6b9d2469201ad8284aa6a7"
server_key_iv: "d83aee8b4df9557bc71735b73a17d514"
server_key_version: "encryptionkey@0"
I20260808 03:51:09.077349 28829 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260808 03:51:09.080540 28845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:09.081825 28829 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:51:09.081965 28829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "dda29ae6a2ca4edcbb41428c7a476a7f"
format_stamp: "Formatted at 2026-08-08 03:51:09 on dist-test-slave-jlzn"
server_key: "204d8bcc9a6b9d2469201ad8284aa6a7"
server_key_iv: "d83aee8b4df9557bc71735b73a17d514"
server_key_version: "encryptionkey@0"
I20260808 03:51:09.082139 28829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:09.101155 28829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:09.104977 28829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:09.105230 28829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:09.124655 28829 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:40575
I20260808 03:51:09.124684 28942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:40575 every 8 connection(s)
I20260808 03:51:09.125970 28829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:51:09.129585 28943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:09.132055 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 28829
I20260808 03:51:09.132274 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:51:09.132890 27210 external_mini_cluster.cc:1428] Setting key 0a67a1e6b041b70e430a30f202608c8d
I20260808 03:51:09.141556 28943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f: Bootstrap starting.
I20260808 03:51:09.145289 28943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:09.146579 28943 log.cc:824] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f: Log is configured to *not* fsync() on all Append() calls
Aug 08 03:51:09 dist-test-slave-jlzn krb5kdc[28810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:09.149421 28943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f: No bootstrap required, opened a new log
I20260808 03:51:09.155334 28946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:09.134785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54550 (local address 127.26.146.190:40575)
0808 03:51:09.135535 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:09.135547 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:09.135586 (+ 39us) server_negotiation.cc:407] Connection header received
0808 03:51:09.137322 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:09.137344 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:09.137696 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:09.138021 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:09.139832 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:09.141084 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:09.142751 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:09.143398 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:09.143517 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:09.148246 (+ 4729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:09.148254 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:09.148268 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:09.148305 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:09.151357 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:09.152099 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:09.152105 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:09.152111 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:09.152291 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:09.152653 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:09.152657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:09.152659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:09.153200 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:09.153515 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:09.153924 (+ 409us) server_negotiation.cc:299] Negotiation successful
0808 03:51:09.154331 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":348,"threads_started":1}
I20260808 03:51:09.157370 28943 raft_consensus.cc:348] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda29ae6a2ca4edcbb41428c7a476a7f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 40575 } }
I20260808 03:51:09.157806 28943 raft_consensus.cc:374] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:09.157928 28943 raft_consensus.cc:729] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dda29ae6a2ca4edcbb41428c7a476a7f, State: Initialized, Role: FOLLOWER
I20260808 03:51:09.158439 28943 consensus_queue.cc:260] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda29ae6a2ca4edcbb41428c7a476a7f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 40575 } }
I20260808 03:51:09.158648 28943 raft_consensus.cc:388] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:51:09.158782 28943 raft_consensus.cc:482] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:51:09.158927 28943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:09.161181 28943 raft_consensus.cc:504] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda29ae6a2ca4edcbb41428c7a476a7f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 40575 } }
I20260808 03:51:09.161633 28943 leader_election.cc:302] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dda29ae6a2ca4edcbb41428c7a476a7f; no voters:
I20260808 03:51:09.162166 28943 leader_election.cc:288] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:51:09.162292 28948 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:09.162935 28948 raft_consensus.cc:686] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f [term 1 LEADER]: Becoming Leader. State: Replica: dda29ae6a2ca4edcbb41428c7a476a7f, State: Running, Role: LEADER
I20260808 03:51:09.163450 28948 consensus_queue.cc:237] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda29ae6a2ca4edcbb41428c7a476a7f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 40575 } }
I20260808 03:51:09.164113 28943 sys_catalog.cc:564] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:51:09.166363 28951 sys_catalog.cc:455] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader dda29ae6a2ca4edcbb41428c7a476a7f. Latest consensus state: current_term: 1 leader_uuid: "dda29ae6a2ca4edcbb41428c7a476a7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda29ae6a2ca4edcbb41428c7a476a7f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 40575 } } }
I20260808 03:51:09.166687 28951 sys_catalog.cc:458] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:09.167304 28949 sys_catalog.cc:455] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dda29ae6a2ca4edcbb41428c7a476a7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda29ae6a2ca4edcbb41428c7a476a7f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 40575 } } }
I20260808 03:51:09.167505 28957 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:51:09.167626 28949 sys_catalog.cc:458] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:09.172678 28957 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:51:09.182893 28957 catalog_manager.cc:1347] Generated new cluster ID: 69b9f1c260764e399d84b7f8c18f8ff9
I20260808 03:51:09.183099 28957 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:51:09.194305 28957 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:51:09.195709 28957 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:51:09.204885 28957 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f: Generated new TSK 0
I20260808 03:51:09.205922 28957 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
I20260808 03:51:09.265125 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:40575
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:09.595034 28974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:09.595324 28974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:09.595466 28974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:09.602645 28974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:09.602762 28974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:09.602886 28974 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 03:51:09.602988 28974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:51:09.612965 28974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.146.190:40575
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:09.614241 28974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:09.615567 28974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:09.628321 28985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:09.628366 28982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:09.628396 28983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:09.629516 28974 server_base.cc:1034] running on GCE node
I20260808 03:51:09.630106 28974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:09.630966 28974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:09.632177 28974 hybrid_clock.cc:648] HybridClock initialized: now 1786161069632104 us; error 41 us; skew 500 ppm
I20260808 03:51:09.632454 28974 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:09 dist-test-slave-jlzn krb5kdc[28810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161069, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:09.638374 28974 init.cc:377] Logged in from keytab as kudu/127.26.146.129@KRBTEST.COM (short username kudu)
I20260808 03:51:09.639851 28974 webserver.cc:460] Webserver started at http://127.26.146.129:40201/ using document root <none> and password file <none>
I20260808 03:51:09.640544 28974 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:09.640648 28974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:09.640960 28974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:09.643927 28974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "abb3892fcf19418fa4b3a8b6328bbf6d"
format_stamp: "Formatted at 2026-08-08 03:51:09 on dist-test-slave-jlzn"
server_key: "3e32be7ca423512be12b2836ba2efaff"
server_key_iv: "06dd581d5a98eb6dd95db3ebf0358e6d"
server_key_version: "encryptionkey@0"
I20260808 03:51:09.644696 28974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "abb3892fcf19418fa4b3a8b6328bbf6d"
format_stamp: "Formatted at 2026-08-08 03:51:09 on dist-test-slave-jlzn"
server_key: "3e32be7ca423512be12b2836ba2efaff"
server_key_iv: "06dd581d5a98eb6dd95db3ebf0358e6d"
server_key_version: "encryptionkey@0"
I20260808 03:51:09.649672 28974 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:51:09.652941 28995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:09.654167 28974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:51:09.654342 28974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "abb3892fcf19418fa4b3a8b6328bbf6d"
format_stamp: "Formatted at 2026-08-08 03:51:09 on dist-test-slave-jlzn"
server_key: "3e32be7ca423512be12b2836ba2efaff"
server_key_iv: "06dd581d5a98eb6dd95db3ebf0358e6d"
server_key_version: "encryptionkey@0"
I20260808 03:51:09.654570 28974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:09.668140 28974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:09.672200 28974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:09.672441 28974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:09.675150 28974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:09.675254 28974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:09.675406 28974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:09.675498 28974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 03:51:09 dist-test-slave-jlzn krb5kdc[28810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161069, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:09.695420 29010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:09.675581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:57655 (local address 127.26.146.190:40575)
0808 03:51:09.676385 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:09.676389 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:09.677172 (+ 783us) server_negotiation.cc:407] Connection header received
0808 03:51:09.679172 (+ 2000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:09.679177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:09.679259 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:09.679352 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:09.681399 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:09.682517 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:09.684944 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:09.685652 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:09.685826 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:09.689559 (+ 3733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:09.689564 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:09.689567 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:09.689601 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:09.692016 (+ 2415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:09.693019 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:09.693025 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:09.693029 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:09.693211 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:09.693690 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:09.693695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:09.693699 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:09.693969 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:09.694103 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:09.694850 (+ 747us) server_negotiation.cc:299] Negotiation successful
0808 03:51:09.695066 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":611,"threads_started":1}
I20260808 03:51:09.696731 29011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:09.676198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:40575 (local address 127.26.146.129:57655)
0808 03:51:09.677018 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:09.677060 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:09.678875 (+ 1815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:09.679594 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:09.679601 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:09.680347 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:09.681184 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:09.681213 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:09.682714 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:09.682720 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:09.684799 (+ 2079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:09.684812 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:09.685900 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:09.685913 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:09.686152 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:09.686802 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:09.686823 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:09.689320 (+ 2497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:09.692152 (+ 2832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:09.692161 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:09.692181 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:09.692818 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:09.693336 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:09.693341 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:09.693348 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:09.693555 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:09.694096 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:09.694104 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:09.694502 (+ 398us) client_negotiation.cc:769] Sending connection context
0808 03:51:09.694724 (+ 222us) client_negotiation.cc:240] Negotiation successful
0808 03:51:09.695218 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":360,"threads_started":1}
I20260808 03:51:09.715458 29010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:09.702159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:54491 (local address 127.26.146.190:40575)
0808 03:51:09.702299 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:09.702303 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:09.702927 (+ 624us) server_negotiation.cc:407] Connection header received
0808 03:51:09.703078 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:09.703083 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:09.703164 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:09.703257 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:09.704652 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:09.705535 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:09.707361 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:09.707961 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:09.708072 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:09.710450 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:09.710456 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:09.710460 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:09.710492 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:09.712980 (+ 2488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:09.713582 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:09.713587 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:09.713591 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:09.713717 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:09.714067 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:09.714071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:09.714075 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:09.714329 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:09.714456 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:09.714904 (+ 448us) server_negotiation.cc:299] Negotiation successful
0808 03:51:09.715134 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 03:51:09.715525 29071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:09.702200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:40575 (local address 127.26.146.129:54491)
0808 03:51:09.702824 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:09.702843 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:09.702971 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:09.703323 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:09.703328 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:09.704086 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:09.704507 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:09.704519 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:09.705674 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:09.705679 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:09.707160 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:09.707177 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:09.708323 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:09.708331 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:09.708514 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:09.709234 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:09.709254 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:09.710281 (+ 1027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:09.713099 (+ 2818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:09.713105 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:09.713109 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:09.713474 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:09.713835 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:09.713840 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:09.713844 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:09.713968 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:09.714441 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:09.714446 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:09.714661 (+ 215us) client_negotiation.cc:769] Sending connection context
0808 03:51:09.714778 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 03:51:09.715075 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":338,"threads_started":1}
I20260808 03:51:09.730232 28974 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:43111
I20260808 03:51:09.730284 29172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:43111 every 8 connection(s)
I20260808 03:51:09.731555 28974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:51:09.733713 29174 heartbeater.cc:347] Connected to a master server at 127.26.146.190:40575
I20260808 03:51:09.733916 29174 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:09.734552 29174 heartbeater.cc:511] Master 127.26.146.190:40575 requested a full tablet report, sending...
I20260808 03:51:09.735314 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 28974
I20260808 03:51:09.735530 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:51:09.736233 27210 external_mini_cluster.cc:1428] Setting key 141894568e097b01cb01021c9004d0d5
I20260808 03:51:09.736748 28869 ts_manager.cc:194] Registered new tserver with Master: abb3892fcf19418fa4b3a8b6328bbf6d (127.26.146.129:43111)
I20260808 03:51:09.738463 28869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.129@KRBTEST.COM'} at 127.26.146.129:57655
WARNING: no policy specified for kudu/127.26.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
I20260808 03:51:09.778481 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:40575
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:10.094422 29178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:10.094705 29178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:10.094810 29178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:10.102010 29178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:10.102131 29178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:10.102221 29178 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 03:51:10.102300 29178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:51:10.111858 29178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.146.190:40575
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:10.113086 29178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:10.114360 29178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:10.126744 29185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:10.126864 29184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:10.127339 29187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:10.127561 29178 server_base.cc:1034] running on GCE node
I20260808 03:51:10.128378 29178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:10.129403 29178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:10.130652 29178 hybrid_clock.cc:648] HybridClock initialized: now 1786161070130580 us; error 41 us; skew 500 ppm
I20260808 03:51:10.130987 29178 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:10 dist-test-slave-jlzn krb5kdc[28810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161070, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:10.134688 29178 init.cc:377] Logged in from keytab as kudu/127.26.146.130@KRBTEST.COM (short username kudu)
I20260808 03:51:10.136335 29178 webserver.cc:460] Webserver started at http://127.26.146.130:36399/ using document root <none> and password file <none>
I20260808 03:51:10.137163 29178 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:10.137277 29178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:10.137596 29178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:10.141520 29178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce"
format_stamp: "Formatted at 2026-08-08 03:51:10 on dist-test-slave-jlzn"
server_key: "75810a1707b87c1093215155426e7312"
server_key_iv: "3279e58cd60717fed1a8689a9138fbb1"
server_key_version: "encryptionkey@0"
I20260808 03:51:10.142472 29178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce"
format_stamp: "Formatted at 2026-08-08 03:51:10 on dist-test-slave-jlzn"
server_key: "75810a1707b87c1093215155426e7312"
server_key_iv: "3279e58cd60717fed1a8689a9138fbb1"
server_key_version: "encryptionkey@0"
I20260808 03:51:10.147898 29178 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 03:51:10.151113 29203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:10.152401 29178 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 03:51:10.152540 29178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce"
format_stamp: "Formatted at 2026-08-08 03:51:10 on dist-test-slave-jlzn"
server_key: "75810a1707b87c1093215155426e7312"
server_key_iv: "3279e58cd60717fed1a8689a9138fbb1"
server_key_version: "encryptionkey@0"
I20260808 03:51:10.152735 29178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:10.172252 29178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:10.176033 29178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:10.176275 29178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:10.178812 29178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:10.178911 29178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:10.179044 29178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:10.179116 29178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 03:51:10 dist-test-slave-jlzn krb5kdc[28810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161070, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:10.200095 29010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:10.179396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:37755 (local address 127.26.146.190:40575)
0808 03:51:10.179606 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:10.179613 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:10.181111 (+ 1498us) server_negotiation.cc:407] Connection header received
0808 03:51:10.183489 (+ 2378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:10.183495 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:10.183582 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:10.183701 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:10.185961 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.187136 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.189335 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.189764 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.189877 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:10.193762 (+ 3885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:10.193769 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:10.193774 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:10.193816 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:10.197039 (+ 3223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:10.197741 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:10.197746 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:10.197750 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:10.197912 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:10.198398 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:10.198404 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:10.198407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:10.198703 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:10.198849 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:10.199516 (+ 667us) server_negotiation.cc:299] Negotiation successful
0808 03:51:10.199734 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260808 03:51:10.201354 29214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:10.180070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:40575 (local address 127.26.146.130:37755)
0808 03:51:10.180965 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:10.181007 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:10.183169 (+ 2162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:10.183983 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:10.183994 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:10.184843 (+ 849us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:10.185723 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:10.185754 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.187345 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.187350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:10.189191 (+ 1841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:10.189206 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.189999 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.190008 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:10.190219 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:10.190862 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:10.190888 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:10.193533 (+ 2645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:10.197187 (+ 3654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:10.197196 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:10.197214 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:10.197611 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:10.198044 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:10.198049 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:10.198053 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:10.198285 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:10.198830 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:10.198837 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:10.199169 (+ 332us) client_negotiation.cc:769] Sending connection context
0808 03:51:10.199417 (+ 248us) client_negotiation.cc:240] Negotiation successful
0808 03:51:10.199961 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":376,"threads_started":1}
I20260808 03:51:10.221946 29178 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:40697
I20260808 03:51:10.222012 29374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:40697 every 8 connection(s)
I20260808 03:51:10.223145 29178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:51:10.223991 29318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:10.208833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:40575 (local address 127.26.146.130:56127)
0808 03:51:10.209427 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:10.209442 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:10.209562 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:10.209915 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:10.209921 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:10.210617 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:10.211125 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:10.211141 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.212464 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.212470 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:10.214389 (+ 1919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:10.214403 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.215116 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.215124 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:10.215308 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:10.216187 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:10.216215 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:10.217455 (+ 1240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:10.221081 (+ 3626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:10.221088 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:10.221092 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:10.221682 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:10.222165 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:10.222170 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:10.222174 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:10.222299 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:10.222855 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:10.222859 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:10.223033 (+ 174us) client_negotiation.cc:769] Sending connection context
0808 03:51:10.223171 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 03:51:10.223491 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":354,"threads_started":1}
I20260808 03:51:10.224180 29010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:10.208731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:56127 (local address 127.26.146.190:40575)
0808 03:51:10.208893 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:10.208898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:10.209521 (+ 623us) server_negotiation.cc:407] Connection header received
0808 03:51:10.209677 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:10.209683 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:10.209759 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:10.209854 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:10.211285 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.212281 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.214522 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.214924 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.215015 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:10.217693 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:10.217712 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:10.217717 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:10.217758 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:10.220945 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:10.221832 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:10.221837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:10.221840 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:10.222009 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:10.222405 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:10.222409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:10.222411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:10.222736 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:10.222868 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:10.223263 (+ 395us) server_negotiation.cc:299] Negotiation successful
0808 03:51:10.223464 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 03:51:10.225165 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 29178
I20260808 03:51:10.225359 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:51:10.225874 27210 external_mini_cluster.cc:1428] Setting key 5fab203d2d92563ab90b7b7f68445938
I20260808 03:51:10.225677 29375 heartbeater.cc:347] Connected to a master server at 127.26.146.190:40575
I20260808 03:51:10.227154 29375 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:10.228061 29375 heartbeater.cc:511] Master 127.26.146.190:40575 requested a full tablet report, sending...
I20260808 03:51:10.229316 28869 ts_manager.cc:194] Registered new tserver with Master: bd75c3ce1a174b62ae0bf9deef1e17ce (127.26.146.130:40697)
I20260808 03:51:10.230104 28869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.130@KRBTEST.COM'} at 127.26.146.130:37755
WARNING: no policy specified for kudu/127.26.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
I20260808 03:51:10.266952 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:40575
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:10.582077 29381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:10.582360 29381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:10.582456 29381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:10.589880 29381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:10.589987 29381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:10.590076 29381 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 03:51:10.590158 29381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:51:10.600097 29381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.146.190:40575
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:10.601298 29381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:10.602545 29381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:10.615283 29392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:10.615312 29390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:10.615347 29389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:10.616160 29381 server_base.cc:1034] running on GCE node
I20260808 03:51:10.616722 29381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:10.617564 29381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:10.618783 29381 hybrid_clock.cc:648] HybridClock initialized: now 1786161070618707 us; error 52 us; skew 500 ppm
I20260808 03:51:10.619068 29381 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:10 dist-test-slave-jlzn krb5kdc[28810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161070, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:10.622079 29381 init.cc:377] Logged in from keytab as kudu/127.26.146.131@KRBTEST.COM (short username kudu)
I20260808 03:51:10.623383 29381 webserver.cc:460] Webserver started at http://127.26.146.131:43341/ using document root <none> and password file <none>
I20260808 03:51:10.624078 29381 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:10.624181 29381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:10.624502 29381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:10.627249 29381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e"
format_stamp: "Formatted at 2026-08-08 03:51:10 on dist-test-slave-jlzn"
server_key: "deeb2f344952cc9a31070c8ee7fcbf81"
server_key_iv: "5053ae3ffd1002f0f1bcd3aa4dc6025c"
server_key_version: "encryptionkey@0"
I20260808 03:51:10.628062 29381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e"
format_stamp: "Formatted at 2026-08-08 03:51:10 on dist-test-slave-jlzn"
server_key: "deeb2f344952cc9a31070c8ee7fcbf81"
server_key_iv: "5053ae3ffd1002f0f1bcd3aa4dc6025c"
server_key_version: "encryptionkey@0"
I20260808 03:51:10.632893 29381 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 03:51:10.636438 29407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:10.637678 29381 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:51:10.637833 29381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e"
format_stamp: "Formatted at 2026-08-08 03:51:10 on dist-test-slave-jlzn"
server_key: "deeb2f344952cc9a31070c8ee7fcbf81"
server_key_iv: "5053ae3ffd1002f0f1bcd3aa4dc6025c"
server_key_version: "encryptionkey@0"
I20260808 03:51:10.638038 29381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:10.654500 29381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:10.658526 29381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:10.658746 29381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:10.661367 29381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:10.661480 29381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:10.661613 29381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:10.661685 29381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 03:51:10 dist-test-slave-jlzn krb5kdc[28810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161070, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:10.682042 29010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:10.661698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:57943 (local address 127.26.146.190:40575)
0808 03:51:10.661896 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:10.661902 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:10.663298 (+ 1396us) server_negotiation.cc:407] Connection header received
0808 03:51:10.665422 (+ 2124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:10.665428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:10.665524 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:10.665643 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:10.667663 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.668691 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.671158 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.671579 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.671710 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:10.675344 (+ 3634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:10.675349 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:10.675352 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:10.675385 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:10.678310 (+ 2925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:10.679359 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:10.679365 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:10.679368 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:10.679536 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:10.680192 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:10.680197 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:10.680201 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:10.680539 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:10.680665 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:10.681598 (+ 933us) server_negotiation.cc:299] Negotiation successful
0808 03:51:10.681784 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 03:51:10.683548 29421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:10.662194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:40575 (local address 127.26.146.131:57943)
0808 03:51:10.663101 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:10.663145 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:10.665132 (+ 1987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:10.665919 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:10.665933 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:10.666637 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:10.667487 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:10.667513 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.668861 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.668866 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:10.671022 (+ 2156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:10.671037 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.672029 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.672034 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:10.672274 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:10.673023 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:10.673044 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:10.675147 (+ 2103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:10.678496 (+ 3349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:10.678505 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:10.678520 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:10.679211 (+ 691us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:10.679660 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:10.679664 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:10.679668 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:10.680042 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:10.680675 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:10.680684 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:10.681138 (+ 454us) client_negotiation.cc:769] Sending connection context
0808 03:51:10.681450 (+ 312us) client_negotiation.cc:240] Negotiation successful
0808 03:51:10.682006 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":369,"threads_started":1}
I20260808 03:51:10.703719 29010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:10.689430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:48701 (local address 127.26.146.190:40575)
0808 03:51:10.689598 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:10.689603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:10.690337 (+ 734us) server_negotiation.cc:407] Connection header received
0808 03:51:10.690497 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:10.690500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:10.690570 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:10.690646 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:10.691848 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.692769 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.694545 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.695160 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.695300 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:10.697466 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:10.697472 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:10.697477 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:10.697515 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:10.700783 (+ 3268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:10.701552 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:10.701558 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:10.701561 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:10.701734 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:10.702131 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:10.702136 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:10.702140 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:10.702404 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:10.702542 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:10.703104 (+ 562us) server_negotiation.cc:299] Negotiation successful
0808 03:51:10.703317 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 03:51:10.703886 29495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:10.689520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:40575 (local address 127.26.146.131:48701)
0808 03:51:10.690226 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:10.690247 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:10.690395 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:10.690691 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:10.690696 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:10.691297 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:10.691694 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:10.691708 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.692903 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.692912 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:10.694391 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:10.694407 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.695386 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.695393 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:10.695554 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:10.696221 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:10.696246 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:10.697254 (+ 1008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:10.700911 (+ 3657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:10.700917 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:10.700921 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:10.701429 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:10.701857 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:10.701862 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:10.701865 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:10.702021 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:10.702607 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:10.702616 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:10.702829 (+ 213us) client_negotiation.cc:769] Sending connection context
0808 03:51:10.702958 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 03:51:10.703317 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":409,"threads_started":1}
I20260808 03:51:10.713214 29381 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:33885
I20260808 03:51:10.713236 29577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:33885 every 8 connection(s)
I20260808 03:51:10.714336 29381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:51:10.716274 29578 heartbeater.cc:347] Connected to a master server at 127.26.146.190:40575
I20260808 03:51:10.716482 29578 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:10.717082 29578 heartbeater.cc:511] Master 127.26.146.190:40575 requested a full tablet report, sending...
I20260808 03:51:10.718295 28869 ts_manager.cc:194] Registered new tserver with Master: 0dcf84aa6b2f4b9b8d7993ba1eb8d33e (127.26.146.131:33885)
I20260808 03:51:10.719079 28869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.131@KRBTEST.COM'} at 127.26.146.131:57943
I20260808 03:51:10.724551 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 29381
I20260808 03:51:10.724704 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:51:10.725117 27210 external_mini_cluster.cc:1428] Setting key f4c1051e6378e6b01b2d26a4cdd695ab
I20260808 03:51:10.727771 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:51:10.741734 29174 heartbeater.cc:503] Master 127.26.146.190:40575 was elected leader, sending a full tablet report...
Aug 08 03:51:10 dist-test-slave-jlzn krb5kdc[28810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161070, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 03:51:10 dist-test-slave-jlzn krb5kdc[28810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161070, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:10.777379 29010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:10.764148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54554 (local address 127.26.146.190:40575)
0808 03:51:10.764357 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:10.764364 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:10.764686 (+ 322us) server_negotiation.cc:407] Connection header received
0808 03:51:10.764884 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:10.764890 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:10.764961 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:10.765045 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:10.766462 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.767208 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.768483 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.768924 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.769036 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:10.772623 (+ 3587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:10.772627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:10.772631 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:10.772663 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:10.774880 (+ 2217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:10.775470 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:10.775474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:10.775477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:10.775594 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:10.776001 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:10.776005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:10.776007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:10.776289 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:10.776512 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:10.776830 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 03:51:10.777041 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 03:51:10.781286 28869 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:51:10.788254 28869 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:51:10.809458 29600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:10.800205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:40697 (local address 127.0.0.1:35272)
0808 03:51:10.800822 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:10.800891 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:10.801077 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:10.802254 (+ 1177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:10.802261 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:10.802314 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:10.802981 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:10.802990 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.804380 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.804385 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:10.806556 (+ 2171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:10.806571 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.808380 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.808390 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:10.808589 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:10.808654 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 03:51:10.808794 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 03:51:10.808993 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":92,"spinlock_wait_cycles":7808,"thread_start_us":335,"threads_started":1}
I20260808 03:51:10.809944 29598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:10.799768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35556 (local address 127.26.146.129:43111)
0808 03:51:10.800340 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:10.800349 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:10.801362 (+ 1013us) server_negotiation.cc:407] Connection header received
0808 03:51:10.801645 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:10.801652 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:10.801844 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:10.802002 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:10.803126 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.804789 (+ 1663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.807416 (+ 2627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.808697 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.808859 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:10.809019 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:10.809568 (+ 549us) server_negotiation.cc:299] Negotiation successful
0808 03:51:10.809732 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":245,"threads_started":1}
I20260808 03:51:10.810148 29601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:10.800502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35272 (local address 127.26.146.130:40697)
0808 03:51:10.801461 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:10.801469 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:10.801491 (+ 22us) server_negotiation.cc:407] Connection header received
0808 03:51:10.801596 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:10.801604 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:10.801847 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:10.802031 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:10.803149 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.804263 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.806709 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.807879 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.808056 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:10.808232 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:10.809705 (+ 1473us) server_negotiation.cc:299] Negotiation successful
0808 03:51:10.809873 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":810,"thread_start_us":1807,"threads_started":1}
I20260808 03:51:10.810753 29599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:10.799738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:43111 (local address 127.0.0.1:35556)
0808 03:51:10.800822 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:10.801272 (+ 450us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:10.801470 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:10.802263 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:10.802267 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:10.802306 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:10.802913 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:10.802928 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.805051 (+ 2123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.805060 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:10.807248 (+ 2188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:10.807262 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.809175 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.809181 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:10.809357 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:10.809390 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 03:51:10.810190 (+ 800us) client_negotiation.cc:240] Negotiation successful
0808 03:51:10.810340 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"mutex_wait_us":47,"spinlock_wait_cycles":1408,"thread_start_us":1092,"threads_started":1}
I20260808 03:51:10.813179 29602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:10.801021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:33885 (local address 127.0.0.1:47738)
0808 03:51:10.801555 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:10.801571 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:10.801690 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:10.805671 (+ 3981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:10.805676 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:10.805712 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:10.806261 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:10.806276 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.807881 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.807886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:10.810775 (+ 2889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:10.810790 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.812479 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.812485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:10.812649 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:10.812680 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 03:51:10.812770 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 03:51:10.812892 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":760,"threads_started":1}
I20260808 03:51:10.813309 29603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:10.801605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47738 (local address 127.26.146.131:33885)
0808 03:51:10.804859 (+ 3254us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:10.804866 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:10.804895 (+ 29us) server_negotiation.cc:407] Connection header received
0808 03:51:10.804984 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:10.804991 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:10.805210 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:10.805389 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:10.806494 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.807719 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.810945 (+ 3226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:10.812278 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:10.812472 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:10.812630 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:10.812844 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 03:51:10.812991 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3075,"thread_start_us":2891,"threads_started":1}
I20260808 03:51:10.814147 29276 tablet_service.cc:1467] Processing CreateTablet for tablet cbed3c89d2404d248213bac2ef41a895 (DEFAULT_TABLE table=test-table [id=399a9eae16204a9db65b423149b4e03a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:10.814245 29076 tablet_service.cc:1467] Processing CreateTablet for tablet cbed3c89d2404d248213bac2ef41a895 (DEFAULT_TABLE table=test-table [id=399a9eae16204a9db65b423149b4e03a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:10.815452 29076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbed3c89d2404d248213bac2ef41a895. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:10.815521 29276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbed3c89d2404d248213bac2ef41a895. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:10.816756 29478 tablet_service.cc:1467] Processing CreateTablet for tablet cbed3c89d2404d248213bac2ef41a895 (DEFAULT_TABLE table=test-table [id=399a9eae16204a9db65b423149b4e03a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:10.818085 29478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbed3c89d2404d248213bac2ef41a895. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:10.826589 29604 tablet_bootstrap.cc:492] T cbed3c89d2404d248213bac2ef41a895 P abb3892fcf19418fa4b3a8b6328bbf6d: Bootstrap starting.
I20260808 03:51:10.829313 29606 tablet_bootstrap.cc:492] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce: Bootstrap starting.
I20260808 03:51:10.831040 29604 tablet_bootstrap.cc:654] T cbed3c89d2404d248213bac2ef41a895 P abb3892fcf19418fa4b3a8b6328bbf6d: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:10.831743 29607 tablet_bootstrap.cc:492] T cbed3c89d2404d248213bac2ef41a895 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: Bootstrap starting.
I20260808 03:51:10.832888 29604 log.cc:824] T cbed3c89d2404d248213bac2ef41a895 P abb3892fcf19418fa4b3a8b6328bbf6d: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:10.833036 29606 tablet_bootstrap.cc:654] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:10.834116 29606 log.cc:824] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:10.834666 29607 tablet_bootstrap.cc:654] T cbed3c89d2404d248213bac2ef41a895 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:10.835764 29607 log.cc:824] T cbed3c89d2404d248213bac2ef41a895 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:10.835884 29604 tablet_bootstrap.cc:492] T cbed3c89d2404d248213bac2ef41a895 P abb3892fcf19418fa4b3a8b6328bbf6d: No bootstrap required, opened a new log
I20260808 03:51:10.836180 29604 ts_tablet_manager.cc:1397] T cbed3c89d2404d248213bac2ef41a895 P abb3892fcf19418fa4b3a8b6328bbf6d: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260808 03:51:10.837116 29606 tablet_bootstrap.cc:492] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce: No bootstrap required, opened a new log
I20260808 03:51:10.837385 29606 ts_tablet_manager.cc:1397] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 03:51:10.838359 29607 tablet_bootstrap.cc:492] T cbed3c89d2404d248213bac2ef41a895 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: No bootstrap required, opened a new log
I20260808 03:51:10.838583 29607 ts_tablet_manager.cc:1397] T cbed3c89d2404d248213bac2ef41a895 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 03:51:10.846411 29607 raft_consensus.cc:348] T cbed3c89d2404d248213bac2ef41a895 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } }
I20260808 03:51:10.846776 29607 raft_consensus.cc:374] T cbed3c89d2404d248213bac2ef41a895 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:10.846889 29607 raft_consensus.cc:729] T cbed3c89d2404d248213bac2ef41a895 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dcf84aa6b2f4b9b8d7993ba1eb8d33e, State: Initialized, Role: FOLLOWER
I20260808 03:51:10.847491 29607 consensus_queue.cc:260] T cbed3c89d2404d248213bac2ef41a895 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } }
I20260808 03:51:10.847478 29606 raft_consensus.cc:348] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } }
I20260808 03:51:10.847477 29604 raft_consensus.cc:348] T cbed3c89d2404d248213bac2ef41a895 P abb3892fcf19418fa4b3a8b6328bbf6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } }
I20260808 03:51:10.847929 29604 raft_consensus.cc:374] T cbed3c89d2404d248213bac2ef41a895 P abb3892fcf19418fa4b3a8b6328bbf6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:10.847931 29606 raft_consensus.cc:374] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:10.848042 29604 raft_consensus.cc:729] T cbed3c89d2404d248213bac2ef41a895 P abb3892fcf19418fa4b3a8b6328bbf6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abb3892fcf19418fa4b3a8b6328bbf6d, State: Initialized, Role: FOLLOWER
I20260808 03:51:10.848042 29606 raft_consensus.cc:729] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd75c3ce1a174b62ae0bf9deef1e17ce, State: Initialized, Role: FOLLOWER
I20260808 03:51:10.848626 29604 consensus_queue.cc:260] T cbed3c89d2404d248213bac2ef41a895 P abb3892fcf19418fa4b3a8b6328bbf6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } }
I20260808 03:51:10.848629 29606 consensus_queue.cc:260] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } }
I20260808 03:51:10.849809 29375 heartbeater.cc:503] Master 127.26.146.190:40575 was elected leader, sending a full tablet report...
I20260808 03:51:10.850472 29606 ts_tablet_manager.cc:1428] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260808 03:51:10.851511 29578 heartbeater.cc:503] Master 127.26.146.190:40575 was elected leader, sending a full tablet report...
I20260808 03:51:10.852085 29607 ts_tablet_manager.cc:1428] T cbed3c89d2404d248213bac2ef41a895 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: Time spent starting tablet: real 0.013s user 0.006s sys 0.009s
I20260808 03:51:10.853430 29604 ts_tablet_manager.cc:1428] T cbed3c89d2404d248213bac2ef41a895 P abb3892fcf19418fa4b3a8b6328bbf6d: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
W20260808 03:51:10.965795 29584 tablet.cc:2369] T cbed3c89d2404d248213bac2ef41a895 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:51:10.975665 29376 tablet.cc:2369] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:51:10.983790 29175 tablet.cc:2369] T cbed3c89d2404d248213bac2ef41a895 P abb3892fcf19418fa4b3a8b6328bbf6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:51:11.061456 29615 raft_consensus.cc:482] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:51:11.061769 29615 raft_consensus.cc:504] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } }
I20260808 03:51:11.063524 29615 leader_election.cc:288] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abb3892fcf19418fa4b3a8b6328bbf6d (127.26.146.129:43111), 0dcf84aa6b2f4b9b8d7993ba1eb8d33e (127.26.146.131:33885)
I20260808 03:51:11.070358 29598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.064089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:37963 (local address 127.26.146.129:43111)
0808 03:51:11.064320 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:11.064327 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:11.065044 (+ 717us) server_negotiation.cc:407] Connection header received
0808 03:51:11.065229 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:11.065235 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:11.065323 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:11.065415 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:11.066052 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.066799 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.068706 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.069477 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.069642 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.069713 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:11.070011 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 03:51:11.070133 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 03:51:11.070359 29603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.064142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:52331 (local address 127.26.146.131:33885)
0808 03:51:11.064342 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:11.064348 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:11.065249 (+ 901us) server_negotiation.cc:407] Connection header received
0808 03:51:11.065412 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:11.065435 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:11.065509 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:11.065593 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:11.066193 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.066935 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.068692 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.069435 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.069561 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.069633 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:11.069957 (+ 324us) server_negotiation.cc:299] Negotiation successful
0808 03:51:11.070090 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 03:51:11.070472 29623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.064234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:33885 (local address 127.26.146.130:52331)
0808 03:51:11.065134 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:11.065150 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:11.065296 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:11.065637 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:11.065640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:11.065663 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:11.066039 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:11.066051 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.067071 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.067074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:11.068555 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:11.068567 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.069658 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.069665 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:11.069811 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.069835 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 03:51:11.069907 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 03:51:11.070036 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"mutex_wait_us":26,"thread_start_us":447,"threads_started":1}
I20260808 03:51:11.070510 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.063994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:43111 (local address 127.26.146.130:37963)
0808 03:51:11.064932 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:11.064948 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:11.065075 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:11.065443 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:11.065447 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:11.065473 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:11.065907 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:11.065920 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.066925 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.066930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:11.068557 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:11.068569 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.069708 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.069713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:11.069858 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.069881 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 03:51:11.069973 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 03:51:11.070090 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":826,"mutex_wait_us":3,"thread_start_us":537,"threads_started":1}
I20260808 03:51:11.071552 29510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbed3c89d2404d248213bac2ef41a895" candidate_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" is_pre_election: true
I20260808 03:51:11.071579 29109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbed3c89d2404d248213bac2ef41a895" candidate_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" is_pre_election: true
I20260808 03:51:11.072075 29109 raft_consensus.cc:2454] T cbed3c89d2404d248213bac2ef41a895 P abb3892fcf19418fa4b3a8b6328bbf6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd75c3ce1a174b62ae0bf9deef1e17ce in term 0.
I20260808 03:51:11.072072 29510 raft_consensus.cc:2454] T cbed3c89d2404d248213bac2ef41a895 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd75c3ce1a174b62ae0bf9deef1e17ce in term 0.
I20260808 03:51:11.072685 29206 leader_election.cc:302] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abb3892fcf19418fa4b3a8b6328bbf6d, bd75c3ce1a174b62ae0bf9deef1e17ce; no voters:
I20260808 03:51:11.073046 29615 raft_consensus.cc:2781] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:51:11.073202 29615 raft_consensus.cc:482] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:51:11.073303 29615 raft_consensus.cc:3037] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:11.075371 29615 raft_consensus.cc:504] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } }
I20260808 03:51:11.075978 29615 leader_election.cc:288] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce [CANDIDATE]: Term 1 election: Requested vote from peers abb3892fcf19418fa4b3a8b6328bbf6d (127.26.146.129:43111), 0dcf84aa6b2f4b9b8d7993ba1eb8d33e (127.26.146.131:33885)
I20260808 03:51:11.076401 29109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbed3c89d2404d248213bac2ef41a895" candidate_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d"
I20260808 03:51:11.076449 29510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbed3c89d2404d248213bac2ef41a895" candidate_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e"
I20260808 03:51:11.076610 29109 raft_consensus.cc:3037] T cbed3c89d2404d248213bac2ef41a895 P abb3892fcf19418fa4b3a8b6328bbf6d [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:11.076619 29510 raft_consensus.cc:3037] T cbed3c89d2404d248213bac2ef41a895 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:11.079679 29109 raft_consensus.cc:2454] T cbed3c89d2404d248213bac2ef41a895 P abb3892fcf19418fa4b3a8b6328bbf6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd75c3ce1a174b62ae0bf9deef1e17ce in term 1.
I20260808 03:51:11.079679 29510 raft_consensus.cc:2454] T cbed3c89d2404d248213bac2ef41a895 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd75c3ce1a174b62ae0bf9deef1e17ce in term 1.
I20260808 03:51:11.080093 29206 leader_election.cc:302] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abb3892fcf19418fa4b3a8b6328bbf6d, bd75c3ce1a174b62ae0bf9deef1e17ce; no voters:
I20260808 03:51:11.080384 29615 raft_consensus.cc:2781] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:11.080825 29615 raft_consensus.cc:686] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce [term 1 LEADER]: Becoming Leader. State: Replica: bd75c3ce1a174b62ae0bf9deef1e17ce, State: Running, Role: LEADER
I20260808 03:51:11.081262 29615 consensus_queue.cc:237] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } }
I20260808 03:51:11.085599 28869 catalog_manager.cc:5495] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce reported cstate change: term changed from 0 to 1, leader changed from <none> to bd75c3ce1a174b62ae0bf9deef1e17ce (127.26.146.130). New cstate: current_term: 1 leader_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } health_report { overall_health: HEALTHY } } }
I20260808 03:51:11.119418 29601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.114392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35280 (local address 127.26.146.130:40697)
0808 03:51:11.114605 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:11.114610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:11.114628 (+ 18us) server_negotiation.cc:407] Connection header received
0808 03:51:11.114896 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:11.114899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:11.114985 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:11.115099 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:51:11.115403 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.116145 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.117378 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.117765 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.117850 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.118285 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:51:11.118858 (+ 573us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:51:11.118962 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:11.119042 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 03:51:11.119109 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 03:51:11.128077 29510 raft_consensus.cc:1261] T cbed3c89d2404d248213bac2ef41a895 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 1 FOLLOWER]: Refusing update from remote peer bd75c3ce1a174b62ae0bf9deef1e17ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:51:11.128087 29109 raft_consensus.cc:1261] T cbed3c89d2404d248213bac2ef41a895 P abb3892fcf19418fa4b3a8b6328bbf6d [term 1 FOLLOWER]: Refusing update from remote peer bd75c3ce1a174b62ae0bf9deef1e17ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:51:11.129138 29624 consensus_queue.cc:1035] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:51:11.129567 29615 consensus_queue.cc:1035] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:51:11.171119 28869 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:51:11.171379 28869 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:51:11.175244 28869 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f: Sending DeleteTablet for 3 replicas of tablet cbed3c89d2404d248213bac2ef41a895
I20260808 03:51:11.176661 29478 tablet_service.cc:1514] Processing DeleteTablet for tablet cbed3c89d2404d248213bac2ef41a895 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:11 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:47738
I20260808 03:51:11.176666 29076 tablet_service.cc:1514] Processing DeleteTablet for tablet cbed3c89d2404d248213bac2ef41a895 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:11 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:35556
I20260808 03:51:11.176940 29276 tablet_service.cc:1514] Processing DeleteTablet for tablet cbed3c89d2404d248213bac2ef41a895 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:11 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:35272
I20260808 03:51:11.181051 28869 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:51:11.181957 28869 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:51:11.189194 29649 tablet_replica.cc:330] T cbed3c89d2404d248213bac2ef41a895 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: stopping tablet replica
I20260808 03:51:11.189733 29649 raft_consensus.cc:2229] T cbed3c89d2404d248213bac2ef41a895 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:51:11.190284 29649 raft_consensus.cc:2258] T cbed3c89d2404d248213bac2ef41a895 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:11.190356 29275 tablet_service.cc:1467] Processing CreateTablet for tablet 4b762e765f74422885e822e73f8ef9e1 (DEFAULT_TABLE table=test-table [id=8e9f1eaa112d481c9e8a5c1145302bb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:11.191097 29275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b762e765f74422885e822e73f8ef9e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:11.192852 29477 tablet_service.cc:1467] Processing CreateTablet for tablet 4b762e765f74422885e822e73f8ef9e1 (DEFAULT_TABLE table=test-table [id=8e9f1eaa112d481c9e8a5c1145302bb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:11.192878 29075 tablet_service.cc:1467] Processing CreateTablet for tablet 4b762e765f74422885e822e73f8ef9e1 (DEFAULT_TABLE table=test-table [id=8e9f1eaa112d481c9e8a5c1145302bb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:11.193317 29477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b762e765f74422885e822e73f8ef9e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:11.193380 29075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b762e765f74422885e822e73f8ef9e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:11.199721 29604 tablet_bootstrap.cc:492] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d: Bootstrap starting.
I20260808 03:51:11.201373 29604 tablet_bootstrap.cc:654] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:11.203923 29606 tablet_bootstrap.cc:492] T 4b762e765f74422885e822e73f8ef9e1 P bd75c3ce1a174b62ae0bf9deef1e17ce: Bootstrap starting.
I20260808 03:51:11.205788 29606 tablet_bootstrap.cc:654] T 4b762e765f74422885e822e73f8ef9e1 P bd75c3ce1a174b62ae0bf9deef1e17ce: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:11.208317 29607 tablet_bootstrap.cc:492] T 4b762e765f74422885e822e73f8ef9e1 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: Bootstrap starting.
I20260808 03:51:11.210192 29607 tablet_bootstrap.cc:654] T 4b762e765f74422885e822e73f8ef9e1 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:11.216723 29649 ts_tablet_manager.cc:1905] T cbed3c89d2404d248213bac2ef41a895 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:11.223975 29607 tablet_bootstrap.cc:492] T 4b762e765f74422885e822e73f8ef9e1 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: No bootstrap required, opened a new log
I20260808 03:51:11.224143 29607 ts_tablet_manager.cc:1397] T 4b762e765f74422885e822e73f8ef9e1 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.007s
I20260808 03:51:11.224931 29607 raft_consensus.cc:348] T 4b762e765f74422885e822e73f8ef9e1 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } }
I20260808 03:51:11.225168 29607 raft_consensus.cc:374] T 4b762e765f74422885e822e73f8ef9e1 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:11.225262 29607 raft_consensus.cc:729] T 4b762e765f74422885e822e73f8ef9e1 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dcf84aa6b2f4b9b8d7993ba1eb8d33e, State: Initialized, Role: FOLLOWER
I20260808 03:51:11.225383 29604 tablet_bootstrap.cc:492] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d: No bootstrap required, opened a new log
I20260808 03:51:11.225589 29604 ts_tablet_manager.cc:1397] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d: Time spent bootstrapping tablet: real 0.026s user 0.005s sys 0.008s
I20260808 03:51:11.225574 29607 consensus_queue.cc:260] T 4b762e765f74422885e822e73f8ef9e1 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } }
I20260808 03:51:11.226289 29607 ts_tablet_manager.cc:1428] T 4b762e765f74422885e822e73f8ef9e1 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 03:51:11.226482 29604 raft_consensus.cc:348] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } }
I20260808 03:51:11.226783 29604 raft_consensus.cc:374] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:11.226923 29604 raft_consensus.cc:729] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abb3892fcf19418fa4b3a8b6328bbf6d, State: Initialized, Role: FOLLOWER
I20260808 03:51:11.227921 29604 consensus_queue.cc:260] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } }
I20260808 03:51:11.228379 29646 tablet_replica.cc:330] T cbed3c89d2404d248213bac2ef41a895 P abb3892fcf19418fa4b3a8b6328bbf6d: stopping tablet replica
I20260808 03:51:11.228605 29606 tablet_bootstrap.cc:492] T 4b762e765f74422885e822e73f8ef9e1 P bd75c3ce1a174b62ae0bf9deef1e17ce: No bootstrap required, opened a new log
I20260808 03:51:11.228760 29606 ts_tablet_manager.cc:1397] T 4b762e765f74422885e822e73f8ef9e1 P bd75c3ce1a174b62ae0bf9deef1e17ce: Time spent bootstrapping tablet: real 0.025s user 0.008s sys 0.005s
I20260808 03:51:11.228864 29646 raft_consensus.cc:2229] T cbed3c89d2404d248213bac2ef41a895 P abb3892fcf19418fa4b3a8b6328bbf6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:51:11.229595 29606 raft_consensus.cc:348] T 4b762e765f74422885e822e73f8ef9e1 P bd75c3ce1a174b62ae0bf9deef1e17ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } }
I20260808 03:51:11.229851 29606 raft_consensus.cc:374] T 4b762e765f74422885e822e73f8ef9e1 P bd75c3ce1a174b62ae0bf9deef1e17ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:11.229964 29606 raft_consensus.cc:729] T 4b762e765f74422885e822e73f8ef9e1 P bd75c3ce1a174b62ae0bf9deef1e17ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd75c3ce1a174b62ae0bf9deef1e17ce, State: Initialized, Role: FOLLOWER
I20260808 03:51:11.230361 29606 consensus_queue.cc:260] T 4b762e765f74422885e822e73f8ef9e1 P bd75c3ce1a174b62ae0bf9deef1e17ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } }
I20260808 03:51:11.230819 29646 raft_consensus.cc:2258] T cbed3c89d2404d248213bac2ef41a895 P abb3892fcf19418fa4b3a8b6328bbf6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:11.229293 29604 ts_tablet_manager.cc:1428] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260808 03:51:11.231269 29606 ts_tablet_manager.cc:1428] T 4b762e765f74422885e822e73f8ef9e1 P bd75c3ce1a174b62ae0bf9deef1e17ce: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 03:51:11.229861 29647 tablet_replica.cc:330] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce: stopping tablet replica
I20260808 03:51:11.233583 29647 raft_consensus.cc:2229] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:51:11.234465 29647 raft_consensus.cc:2258] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:11.236090 29649 ts_tablet_manager.cc:1918] T cbed3c89d2404d248213bac2ef41a895 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:51:11.236363 29649 log.cc:1196] T cbed3c89d2404d248213bac2ef41a895 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/cbed3c89d2404d248213bac2ef41a895
I20260808 03:51:11.237071 29649 ts_tablet_manager.cc:1939] T cbed3c89d2404d248213bac2ef41a895 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: Deleting consensus metadata
I20260808 03:51:11.238972 28854 catalog_manager.cc:4841] TS 0dcf84aa6b2f4b9b8d7993ba1eb8d33e (127.26.146.131:33885): tablet cbed3c89d2404d248213bac2ef41a895 (table test-table [id=399a9eae16204a9db65b423149b4e03a]) successfully deleted
I20260808 03:51:11.239698 29647 ts_tablet_manager.cc:1905] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:11.240207 29646 ts_tablet_manager.cc:1905] T cbed3c89d2404d248213bac2ef41a895 P abb3892fcf19418fa4b3a8b6328bbf6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:11.246929 29646 ts_tablet_manager.cc:1918] T cbed3c89d2404d248213bac2ef41a895 P abb3892fcf19418fa4b3a8b6328bbf6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:51:11.247139 29646 log.cc:1196] T cbed3c89d2404d248213bac2ef41a895 P abb3892fcf19418fa4b3a8b6328bbf6d: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-0/wal/wals/cbed3c89d2404d248213bac2ef41a895
I20260808 03:51:11.247706 29646 ts_tablet_manager.cc:1939] T cbed3c89d2404d248213bac2ef41a895 P abb3892fcf19418fa4b3a8b6328bbf6d: Deleting consensus metadata
I20260808 03:51:11.249001 29647 ts_tablet_manager.cc:1918] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:51:11.249317 29647 log.cc:1196] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/cbed3c89d2404d248213bac2ef41a895
I20260808 03:51:11.249751 28853 catalog_manager.cc:4841] TS abb3892fcf19418fa4b3a8b6328bbf6d (127.26.146.129:43111): tablet cbed3c89d2404d248213bac2ef41a895 (table test-table [id=399a9eae16204a9db65b423149b4e03a]) successfully deleted
I20260808 03:51:11.250079 29647 ts_tablet_manager.cc:1939] T cbed3c89d2404d248213bac2ef41a895 P bd75c3ce1a174b62ae0bf9deef1e17ce: Deleting consensus metadata
I20260808 03:51:11.251370 28848 catalog_manager.cc:4841] TS bd75c3ce1a174b62ae0bf9deef1e17ce (127.26.146.130:40697): tablet cbed3c89d2404d248213bac2ef41a895 (table test-table [id=399a9eae16204a9db65b423149b4e03a]) successfully deleted
I20260808 03:51:11.348280 29616 raft_consensus.cc:482] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:51:11.348533 29616 raft_consensus.cc:504] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } }
I20260808 03:51:11.350522 29616 leader_election.cc:288] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd75c3ce1a174b62ae0bf9deef1e17ce (127.26.146.130:40697), 0dcf84aa6b2f4b9b8d7993ba1eb8d33e (127.26.146.131:33885)
I20260808 03:51:11.358287 29603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.351130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:37625 (local address 127.26.146.131:33885)
0808 03:51:11.351330 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:11.351336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:11.352257 (+ 921us) server_negotiation.cc:407] Connection header received
0808 03:51:11.352405 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:11.352411 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:11.352491 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:11.352580 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:11.353331 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.354324 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.356244 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.357330 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.357484 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.357559 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:11.357894 (+ 335us) server_negotiation.cc:299] Negotiation successful
0808 03:51:11.358023 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 03:51:11.358426 29657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.350813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:33885 (local address 127.26.146.129:37625)
0808 03:51:11.352146 (+ 1333us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:11.352164 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:11.352287 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:11.352764 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:11.352769 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:11.352795 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:11.353157 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:11.353170 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.354447 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.354451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:11.356086 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:11.356099 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.357534 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.357541 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:11.357719 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.357754 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 03:51:11.357855 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 03:51:11.357983 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1222,"mutex_wait_us":3,"thread_start_us":1037,"threads_started":1}
I20260808 03:51:11.359127 29601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.351135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:43183 (local address 127.26.146.130:40697)
0808 03:51:11.351327 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:11.351333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:11.351924 (+ 591us) server_negotiation.cc:407] Connection header received
0808 03:51:11.352249 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:11.352255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:11.352329 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:11.352410 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:11.352937 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.354031 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.356656 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.357855 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.358008 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.358088 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:11.358715 (+ 627us) server_negotiation.cc:299] Negotiation successful
0808 03:51:11.358845 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 03:51:11.359169 29510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b762e765f74422885e822e73f8ef9e1" candidate_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" is_pre_election: true
I20260808 03:51:11.359439 29510 raft_consensus.cc:2454] T 4b762e765f74422885e822e73f8ef9e1 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abb3892fcf19418fa4b3a8b6328bbf6d in term 0.
I20260808 03:51:11.359845 29654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.350743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:40697 (local address 127.26.146.129:43183)
0808 03:51:11.351786 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:11.351840 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:11.352061 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:11.352432 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:11.352436 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:11.352459 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:11.352768 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:11.352778 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.354171 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.354177 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:11.356504 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:11.356521 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.358333 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.358340 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:11.358523 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.358557 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 03:51:11.359401 (+ 844us) client_negotiation.cc:240] Negotiation successful
0808 03:51:11.359498 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":903,"mutex_wait_us":21,"thread_start_us":734,"threads_started":1}
I20260808 03:51:11.360034 29000 leader_election.cc:302] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dcf84aa6b2f4b9b8d7993ba1eb8d33e, abb3892fcf19418fa4b3a8b6328bbf6d; no voters:
I20260808 03:51:11.360450 29616 raft_consensus.cc:2781] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:51:11.360556 29616 raft_consensus.cc:482] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:51:11.360651 29616 raft_consensus.cc:3037] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:11.360734 29310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b762e765f74422885e822e73f8ef9e1" candidate_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" is_pre_election: true
I20260808 03:51:11.361189 29310 raft_consensus.cc:2454] T 4b762e765f74422885e822e73f8ef9e1 P bd75c3ce1a174b62ae0bf9deef1e17ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abb3892fcf19418fa4b3a8b6328bbf6d in term 0.
I20260808 03:51:11.363327 29616 raft_consensus.cc:504] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } }
I20260808 03:51:11.363935 29616 leader_election.cc:288] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d [CANDIDATE]: Term 1 election: Requested vote from peers bd75c3ce1a174b62ae0bf9deef1e17ce (127.26.146.130:40697), 0dcf84aa6b2f4b9b8d7993ba1eb8d33e (127.26.146.131:33885)
I20260808 03:51:11.364373 29310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b762e765f74422885e822e73f8ef9e1" candidate_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce"
I20260808 03:51:11.364478 29510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b762e765f74422885e822e73f8ef9e1" candidate_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e"
I20260808 03:51:11.364576 29310 raft_consensus.cc:3037] T 4b762e765f74422885e822e73f8ef9e1 P bd75c3ce1a174b62ae0bf9deef1e17ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:11.364694 29510 raft_consensus.cc:3037] T 4b762e765f74422885e822e73f8ef9e1 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:11.367089 29310 raft_consensus.cc:2454] T 4b762e765f74422885e822e73f8ef9e1 P bd75c3ce1a174b62ae0bf9deef1e17ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abb3892fcf19418fa4b3a8b6328bbf6d in term 1.
I20260808 03:51:11.367671 29510 raft_consensus.cc:2454] T 4b762e765f74422885e822e73f8ef9e1 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abb3892fcf19418fa4b3a8b6328bbf6d in term 1.
I20260808 03:51:11.368031 28996 leader_election.cc:302] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abb3892fcf19418fa4b3a8b6328bbf6d, bd75c3ce1a174b62ae0bf9deef1e17ce; no voters:
I20260808 03:51:11.368327 29616 raft_consensus.cc:2781] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:11.368806 29616 raft_consensus.cc:686] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d [term 1 LEADER]: Becoming Leader. State: Replica: abb3892fcf19418fa4b3a8b6328bbf6d, State: Running, Role: LEADER
I20260808 03:51:11.369220 29616 consensus_queue.cc:237] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } }
I20260808 03:51:11.372563 28868 catalog_manager.cc:5495] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d reported cstate change: term changed from 0 to 1, leader changed from <none> to abb3892fcf19418fa4b3a8b6328bbf6d (127.26.146.129). New cstate: current_term: 1 leader_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } health_report { overall_health: UNKNOWN } } }
Aug 08 03:51:11 dist-test-slave-jlzn krb5kdc[28810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161069, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:11.452350 29669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.438130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54566 (local address 127.26.146.190:40575)
0808 03:51:11.439039 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:11.439043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:11.439060 (+ 17us) server_negotiation.cc:407] Connection header received
0808 03:51:11.439100 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:11.439103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:11.439154 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:11.439239 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:11.440369 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.441176 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.442440 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.442841 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.442947 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.446735 (+ 3788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:11.446740 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:11.446744 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:11.446773 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:11.449319 (+ 2546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:11.450201 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:11.450205 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:11.450207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:11.450317 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:11.450760 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:11.450763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:11.450765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:11.451032 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:11.451137 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:11.451713 (+ 576us) server_negotiation.cc:299] Negotiation successful
0808 03:51:11.452055 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":168,"server-negotiator.queue_time_us":796,"thread_start_us":658,"threads_started":1}
I20260808 03:51:11.452415 29668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.438057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:40575 (local address 127.0.0.1:54566)
0808 03:51:11.438695 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:11.438716 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:11.438875 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:11.439298 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:11.439302 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:11.439998 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:11.440214 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:11.440224 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.441302 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.441305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:11.442308 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:11.442321 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.442992 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.442998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:11.443179 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.443865 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:11.443883 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:11.446514 (+ 2631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:11.449459 (+ 2945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:11.449472 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:11.449480 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:11.450090 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:11.450422 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:11.450427 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:11.450430 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:11.450658 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:11.451167 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:11.451176 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:11.451415 (+ 239us) client_negotiation.cc:769] Sending connection context
0808 03:51:11.451619 (+ 204us) client_negotiation.cc:240] Negotiation successful
0808 03:51:11.451947 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":338,"threads_started":1}
I20260808 03:51:11.458212 28868 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:54566:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 03:51:11.459523 28868 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:51:11.466554 29076 tablet_service.cc:1467] Processing CreateTablet for tablet 8e3e1f019b534feca8152e3f5ac13c9a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=04e220a2865a4509a64814e7b750d632]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 03:51:11.466794 29276 tablet_service.cc:1467] Processing CreateTablet for tablet 8e3e1f019b534feca8152e3f5ac13c9a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=04e220a2865a4509a64814e7b750d632]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 03:51:11.467101 29076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e3e1f019b534feca8152e3f5ac13c9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:11.467407 29276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e3e1f019b534feca8152e3f5ac13c9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:11.471226 29477 tablet_service.cc:1467] Processing CreateTablet for tablet 8e3e1f019b534feca8152e3f5ac13c9a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=04e220a2865a4509a64814e7b750d632]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 03:51:11.471747 29477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e3e1f019b534feca8152e3f5ac13c9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:11.472918 29604 tablet_bootstrap.cc:492] T 8e3e1f019b534feca8152e3f5ac13c9a P abb3892fcf19418fa4b3a8b6328bbf6d: Bootstrap starting.
I20260808 03:51:11.474550 29604 tablet_bootstrap.cc:654] T 8e3e1f019b534feca8152e3f5ac13c9a P abb3892fcf19418fa4b3a8b6328bbf6d: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:11.475701 29606 tablet_bootstrap.cc:492] T 8e3e1f019b534feca8152e3f5ac13c9a P bd75c3ce1a174b62ae0bf9deef1e17ce: Bootstrap starting.
I20260808 03:51:11.477458 29606 tablet_bootstrap.cc:654] T 8e3e1f019b534feca8152e3f5ac13c9a P bd75c3ce1a174b62ae0bf9deef1e17ce: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:11.478806 29607 tablet_bootstrap.cc:492] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: Bootstrap starting.
I20260808 03:51:11.480510 29607 tablet_bootstrap.cc:654] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:11.484380 29607 tablet_bootstrap.cc:492] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: No bootstrap required, opened a new log
I20260808 03:51:11.484547 29607 ts_tablet_manager.cc:1397] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260808 03:51:11.485349 29607 raft_consensus.cc:348] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } }
I20260808 03:51:11.485620 29604 tablet_bootstrap.cc:492] T 8e3e1f019b534feca8152e3f5ac13c9a P abb3892fcf19418fa4b3a8b6328bbf6d: No bootstrap required, opened a new log
I20260808 03:51:11.485587 29607 raft_consensus.cc:374] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:11.485684 29607 raft_consensus.cc:729] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dcf84aa6b2f4b9b8d7993ba1eb8d33e, State: Initialized, Role: FOLLOWER
I20260808 03:51:11.485812 29604 ts_tablet_manager.cc:1397] T 8e3e1f019b534feca8152e3f5ac13c9a P abb3892fcf19418fa4b3a8b6328bbf6d: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260808 03:51:11.485934 29607 consensus_queue.cc:260] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } }
I20260808 03:51:11.486501 29604 raft_consensus.cc:348] T 8e3e1f019b534feca8152e3f5ac13c9a P abb3892fcf19418fa4b3a8b6328bbf6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } }
I20260808 03:51:11.486743 29604 raft_consensus.cc:374] T 8e3e1f019b534feca8152e3f5ac13c9a P abb3892fcf19418fa4b3a8b6328bbf6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:11.486817 29604 raft_consensus.cc:729] T 8e3e1f019b534feca8152e3f5ac13c9a P abb3892fcf19418fa4b3a8b6328bbf6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abb3892fcf19418fa4b3a8b6328bbf6d, State: Initialized, Role: FOLLOWER
I20260808 03:51:11.486883 29606 tablet_bootstrap.cc:492] T 8e3e1f019b534feca8152e3f5ac13c9a P bd75c3ce1a174b62ae0bf9deef1e17ce: No bootstrap required, opened a new log
I20260808 03:51:11.486732 29614 tablet_replica.cc:439] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } } }
I20260808 03:51:11.487037 29614 tablet_replica.cc:442] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 03:51:11.487061 29606 ts_tablet_manager.cc:1397] T 8e3e1f019b534feca8152e3f5ac13c9a P bd75c3ce1a174b62ae0bf9deef1e17ce: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260808 03:51:11.487035 29604 consensus_queue.cc:260] T 8e3e1f019b534feca8152e3f5ac13c9a P abb3892fcf19418fa4b3a8b6328bbf6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } }
I20260808 03:51:11.487565 29604 ts_tablet_manager.cc:1428] T 8e3e1f019b534feca8152e3f5ac13c9a P abb3892fcf19418fa4b3a8b6328bbf6d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260808 03:51:11.487861 29616 tablet_replica.cc:439] T 8e3e1f019b534feca8152e3f5ac13c9a P abb3892fcf19418fa4b3a8b6328bbf6d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } } }
I20260808 03:51:11.488150 29616 tablet_replica.cc:442] T 8e3e1f019b534feca8152e3f5ac13c9a P abb3892fcf19418fa4b3a8b6328bbf6d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 03:51:11.488310 29607 ts_tablet_manager.cc:1428] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260808 03:51:11.489018 29606 raft_consensus.cc:348] T 8e3e1f019b534feca8152e3f5ac13c9a P bd75c3ce1a174b62ae0bf9deef1e17ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } }
I20260808 03:51:11.489255 29606 raft_consensus.cc:374] T 8e3e1f019b534feca8152e3f5ac13c9a P bd75c3ce1a174b62ae0bf9deef1e17ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:11.489388 29606 raft_consensus.cc:729] T 8e3e1f019b534feca8152e3f5ac13c9a P bd75c3ce1a174b62ae0bf9deef1e17ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd75c3ce1a174b62ae0bf9deef1e17ce, State: Initialized, Role: FOLLOWER
I20260808 03:51:11.489660 29606 consensus_queue.cc:260] T 8e3e1f019b534feca8152e3f5ac13c9a P bd75c3ce1a174b62ae0bf9deef1e17ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } }
I20260808 03:51:11.490268 29606 ts_tablet_manager.cc:1428] T 8e3e1f019b534feca8152e3f5ac13c9a P bd75c3ce1a174b62ae0bf9deef1e17ce: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 03:51:11.490160 29615 tablet_replica.cc:439] T 8e3e1f019b534feca8152e3f5ac13c9a P bd75c3ce1a174b62ae0bf9deef1e17ce: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } } }
I20260808 03:51:11.490459 29615 tablet_replica.cc:442] T 8e3e1f019b534feca8152e3f5ac13c9a P bd75c3ce1a174b62ae0bf9deef1e17ce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 03:51:11.587833 29614 raft_consensus.cc:482] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:51:11.588122 29614 raft_consensus.cc:504] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } }
I20260808 03:51:11.589951 29614 leader_election.cc:288] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abb3892fcf19418fa4b3a8b6328bbf6d (127.26.146.129:43111), bd75c3ce1a174b62ae0bf9deef1e17ce (127.26.146.130:40697)
I20260808 03:51:11.599258 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.590631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:52179 (local address 127.26.146.129:43111)
0808 03:51:11.592234 (+ 1603us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:11.592241 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:11.593086 (+ 845us) server_negotiation.cc:407] Connection header received
0808 03:51:11.593233 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:11.593239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:11.593318 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:11.593414 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:11.594046 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.594850 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.597445 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.598283 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.598436 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.598523 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:11.598826 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 03:51:11.598964 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1479,"thread_start_us":1323,"threads_started":1}
I20260808 03:51:11.599368 29676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.590208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:43111 (local address 127.26.146.131:52179)
0808 03:51:11.592955 (+ 2747us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:11.592970 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:11.593097 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:11.593452 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:11.593456 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:11.593482 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:11.593910 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:11.593918 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.594972 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.594977 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:11.597253 (+ 2276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:11.597265 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.598436 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.598442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:11.598653 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.598696 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 03:51:11.598788 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 03:51:11.598912 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2644,"mutex_wait_us":93,"thread_start_us":2154,"threads_started":1}
I20260808 03:51:11.600190 29601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.590592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:45141 (local address 127.26.146.130:40697)
0808 03:51:11.590788 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:11.590795 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:11.592785 (+ 1990us) server_negotiation.cc:407] Connection header received
0808 03:51:11.593096 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:11.593102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:11.593178 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:11.593265 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:11.593907 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.595007 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.597422 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.598716 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.598877 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.598947 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:11.599731 (+ 784us) server_negotiation.cc:299] Negotiation successful
0808 03:51:11.599911 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 03:51:11.600469 29109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e3e1f019b534feca8152e3f5ac13c9a" candidate_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" is_pre_election: true
I20260808 03:51:11.600708 29109 raft_consensus.cc:2454] T 8e3e1f019b534feca8152e3f5ac13c9a P abb3892fcf19418fa4b3a8b6328bbf6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dcf84aa6b2f4b9b8d7993ba1eb8d33e in term 0.
I20260808 03:51:11.600960 29677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.590186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:40697 (local address 127.26.146.131:45141)
0808 03:51:11.592663 (+ 2477us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:11.592685 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:11.592911 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:11.593298 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:11.593303 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:11.593328 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:11.593693 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:11.593707 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.595164 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.595169 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:11.597250 (+ 2081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:11.597265 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.599119 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.599126 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:11.599557 (+ 431us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.599591 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 03:51:11.600490 (+ 899us) client_negotiation.cc:240] Negotiation successful
0808 03:51:11.600624 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2338,"mutex_wait_us":27,"thread_start_us":2525,"threads_started":1}
I20260808 03:51:11.601228 29413 leader_election.cc:302] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dcf84aa6b2f4b9b8d7993ba1eb8d33e, abb3892fcf19418fa4b3a8b6328bbf6d; no voters:
I20260808 03:51:11.601651 29614 raft_consensus.cc:2781] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:51:11.601619 29310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e3e1f019b534feca8152e3f5ac13c9a" candidate_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" is_pre_election: true
I20260808 03:51:11.601773 29614 raft_consensus.cc:482] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:51:11.601866 29614 raft_consensus.cc:3037] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:11.601882 29310 raft_consensus.cc:2454] T 8e3e1f019b534feca8152e3f5ac13c9a P bd75c3ce1a174b62ae0bf9deef1e17ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dcf84aa6b2f4b9b8d7993ba1eb8d33e in term 0.
I20260808 03:51:11.604192 29614 raft_consensus.cc:504] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } }
I20260808 03:51:11.604777 29614 leader_election.cc:288] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [CANDIDATE]: Term 1 election: Requested vote from peers abb3892fcf19418fa4b3a8b6328bbf6d (127.26.146.129:43111), bd75c3ce1a174b62ae0bf9deef1e17ce (127.26.146.130:40697)
I20260808 03:51:11.605247 29109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e3e1f019b534feca8152e3f5ac13c9a" candidate_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d"
I20260808 03:51:11.605288 29310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e3e1f019b534feca8152e3f5ac13c9a" candidate_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce"
I20260808 03:51:11.605432 29109 raft_consensus.cc:3037] T 8e3e1f019b534feca8152e3f5ac13c9a P abb3892fcf19418fa4b3a8b6328bbf6d [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:11.605522 29310 raft_consensus.cc:3037] T 8e3e1f019b534feca8152e3f5ac13c9a P bd75c3ce1a174b62ae0bf9deef1e17ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:11.607301 29109 raft_consensus.cc:2454] T 8e3e1f019b534feca8152e3f5ac13c9a P abb3892fcf19418fa4b3a8b6328bbf6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dcf84aa6b2f4b9b8d7993ba1eb8d33e in term 1.
I20260808 03:51:11.607640 29413 leader_election.cc:302] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dcf84aa6b2f4b9b8d7993ba1eb8d33e, abb3892fcf19418fa4b3a8b6328bbf6d; no voters:
I20260808 03:51:11.607939 29614 raft_consensus.cc:2781] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:11.608402 29614 raft_consensus.cc:686] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 1 LEADER]: Becoming Leader. State: Replica: 0dcf84aa6b2f4b9b8d7993ba1eb8d33e, State: Running, Role: LEADER
I20260808 03:51:11.608533 29310 raft_consensus.cc:2454] T 8e3e1f019b534feca8152e3f5ac13c9a P bd75c3ce1a174b62ae0bf9deef1e17ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dcf84aa6b2f4b9b8d7993ba1eb8d33e in term 1.
I20260808 03:51:11.609095 29614 consensus_queue.cc:237] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } }
I20260808 03:51:11.611143 29679 tablet_replica.cc:439] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: TxnStatusTablet state changed. Reason: New leader 0dcf84aa6b2f4b9b8d7993ba1eb8d33e. Latest consensus state: current_term: 1 leader_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } } }
I20260808 03:51:11.611989 29679 tablet_replica.cc:442] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: This TxnStatusTablet replica's current role is: LEADER
I20260808 03:51:11.612679 29682 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 03:51:11.612907 29682 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 03:51:11.612967 28865 catalog_manager.cc:5495] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0dcf84aa6b2f4b9b8d7993ba1eb8d33e (127.26.146.131). New cstate: current_term: 1 leader_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } health_report { overall_health: HEALTHY } } }
I20260808 03:51:11.646783 29668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.639007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:33885 (local address 127.0.0.1:47752)
0808 03:51:11.639307 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:11.639328 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:11.639585 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:11.640110 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:11.640115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:11.640129 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 03:51:11.640476 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:11.640491 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.641750 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.641756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:11.643775 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:11.643791 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.644994 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.645001 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:11.645201 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.645278 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 03:51:11.646230 (+ 952us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 03:51:11.646252 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 03:51:11.646331 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 03:51:11.646485 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 03:51:11.646858 29603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.638927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47752 (local address 127.26.146.131:33885)
0808 03:51:11.639120 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:11.639128 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:11.639402 (+ 274us) server_negotiation.cc:407] Connection header received
0808 03:51:11.639786 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:11.639790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:11.639943 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:11.640076 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:51:11.640613 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.641571 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.643927 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.644837 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.644930 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.645546 (+ 616us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:51:11.646123 (+ 577us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:51:11.646253 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:11.646526 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 03:51:11.646602 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 03:51:11.652293 29109 raft_consensus.cc:1261] T 8e3e1f019b534feca8152e3f5ac13c9a P abb3892fcf19418fa4b3a8b6328bbf6d [term 1 FOLLOWER]: Refusing update from remote peer 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:51:11.652684 29310 raft_consensus.cc:1261] T 8e3e1f019b534feca8152e3f5ac13c9a P bd75c3ce1a174b62ae0bf9deef1e17ce [term 1 FOLLOWER]: Refusing update from remote peer 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:51:11.652923 29679 consensus_queue.cc:1035] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:51:11.653669 29679 consensus_queue.cc:1035] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:51:11.669358 29615 tablet_replica.cc:439] T 8e3e1f019b534feca8152e3f5ac13c9a P bd75c3ce1a174b62ae0bf9deef1e17ce: TxnStatusTablet state changed. Reason: New leader 0dcf84aa6b2f4b9b8d7993ba1eb8d33e. Latest consensus state: current_term: 1 leader_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } } }
I20260808 03:51:11.669633 29615 tablet_replica.cc:442] T 8e3e1f019b534feca8152e3f5ac13c9a P bd75c3ce1a174b62ae0bf9deef1e17ce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 03:51:11.670037 29616 tablet_replica.cc:439] T 8e3e1f019b534feca8152e3f5ac13c9a P abb3892fcf19418fa4b3a8b6328bbf6d: TxnStatusTablet state changed. Reason: New leader 0dcf84aa6b2f4b9b8d7993ba1eb8d33e. Latest consensus state: current_term: 1 leader_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } } }
I20260808 03:51:11.670565 29616 tablet_replica.cc:442] T 8e3e1f019b534feca8152e3f5ac13c9a P abb3892fcf19418fa4b3a8b6328bbf6d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 03:51:11.672384 29688 mvcc.cc:205] Tried to move back new op lower bound from 7316115749480292352 to 7316115749314580480. Current Snapshot: MvccSnapshot[applied={T|T < 7316115749480292352}]
I20260808 03:51:11.678542 29614 tablet_replica.cc:439] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } } }
I20260808 03:51:11.679644 29615 tablet_replica.cc:439] T 8e3e1f019b534feca8152e3f5ac13c9a P bd75c3ce1a174b62ae0bf9deef1e17ce: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } } }
I20260808 03:51:11.680034 29614 tablet_replica.cc:442] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: This TxnStatusTablet replica's current role is: LEADER
I20260808 03:51:11.680097 29615 tablet_replica.cc:442] T 8e3e1f019b534feca8152e3f5ac13c9a P bd75c3ce1a174b62ae0bf9deef1e17ce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 03:51:11.681851 29614 tablet_replica.cc:439] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } } }
I20260808 03:51:11.682267 29614 tablet_replica.cc:442] T 8e3e1f019b534feca8152e3f5ac13c9a P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: This TxnStatusTablet replica's current role is: LEADER
I20260808 03:51:11.686290 29616 tablet_replica.cc:439] T 8e3e1f019b534feca8152e3f5ac13c9a P abb3892fcf19418fa4b3a8b6328bbf6d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb3892fcf19418fa4b3a8b6328bbf6d" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43111 } } peers { permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 } } peers { permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 } } }
I20260808 03:51:11.686683 29616 tablet_replica.cc:442] T 8e3e1f019b534feca8152e3f5ac13c9a P abb3892fcf19418fa4b3a8b6328bbf6d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 03:51:11.698271 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.689181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54198 (local address 127.26.146.129:43111)
0808 03:51:11.689375 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:11.689382 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:11.690307 (+ 925us) server_negotiation.cc:407] Connection header received
0808 03:51:11.690489 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:11.690495 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:11.690643 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:11.690817 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:51:11.691556 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.693049 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.695263 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.696084 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.696213 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.696683 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:51:11.697401 (+ 718us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:51:11.697547 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:11.697714 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 03:51:11.697800 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 03:51:11.718575 29705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.711455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:33885 (local address 127.26.146.129:50029)
0808 03:51:11.712514 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:11.712532 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:11.712671 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:11.712986 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:11.712990 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:11.713002 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 03:51:11.713582 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:11.713595 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.714768 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.714773 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:11.716358 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:11.716372 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.717220 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.717227 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:11.717392 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.717456 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 03:51:11.717962 (+ 506us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 03:51:11.717970 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 03:51:11.718037 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 03:51:11.718169 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":888,"thread_start_us":595,"threads_started":1}
I20260808 03:51:11.721225 29603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.711181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:50029 (local address 127.26.146.131:33885)
0808 03:51:11.711383 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:11.711389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:11.712616 (+ 1227us) server_negotiation.cc:407] Connection header received
0808 03:51:11.712807 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:11.712811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:11.712875 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:11.712947 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:51:11.713825 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.714646 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.716499 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.717021 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.717151 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.717666 (+ 515us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:51:11.717807 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:51:11.717878 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:11.718092 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 03:51:11.718166 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 03:51:11.733587 29510 raft_consensus.cc:1261] T 4b762e765f74422885e822e73f8ef9e1 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 1 FOLLOWER]: Refusing update from remote peer abb3892fcf19418fa4b3a8b6328bbf6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:51:11.734045 29308 raft_consensus.cc:1261] T 4b762e765f74422885e822e73f8ef9e1 P bd75c3ce1a174b62ae0bf9deef1e17ce [term 1 FOLLOWER]: Refusing update from remote peer abb3892fcf19418fa4b3a8b6328bbf6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:51:11.734395 29658 consensus_queue.cc:1035] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dcf84aa6b2f4b9b8d7993ba1eb8d33e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 33885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:51:11.735671 29616 consensus_queue.cc:1035] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd75c3ce1a174b62ae0bf9deef1e17ce" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:51:11.751953 29690 mvcc.cc:205] Tried to move back new op lower bound from 7316115749814452224 to 7316115748332040192. Current Snapshot: MvccSnapshot[applied={T|T < 7316115749814452224}]
I20260808 03:51:11.795245 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.782375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:41981 (local address 127.26.146.129:43111)
0808 03:51:11.782559 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:11.782566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:11.788178 (+ 5612us) server_negotiation.cc:407] Connection header received
0808 03:51:11.788355 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:11.788359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:11.788437 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:11.788511 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:51:11.789389 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.790495 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.792932 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.793537 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.793673 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.794400 (+ 727us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:51:11.794543 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:51:11.794629 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:11.794777 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 03:51:11.794855 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 03:51:11.795562 29710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.781780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:43111 (local address 127.26.146.131:41981)
0808 03:51:11.788057 (+ 6277us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:11.788078 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:11.788217 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:11.788544 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:11.788552 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:11.788564 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 03:51:11.789197 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:11.789213 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.790635 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.790641 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:11.792789 (+ 2148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:11.792806 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.793855 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.793864 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:11.794137 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.794211 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 03:51:11.794651 (+ 440us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 03:51:11.794663 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 03:51:11.794746 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 03:51:11.794881 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6067,"thread_start_us":5659,"threads_started":1}
I20260808 03:51:11.850273 28865 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:51:11.850651 28865 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:51:11.854418 28865 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dda29ae6a2ca4edcbb41428c7a476a7f: Sending DeleteTablet for 3 replicas of tablet 4b762e765f74422885e822e73f8ef9e1
I20260808 03:51:11.855269 29076 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b762e765f74422885e822e73f8ef9e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:11 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:35556
I20260808 03:51:11.855644 29276 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b762e765f74422885e822e73f8ef9e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:11 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:35272
I20260808 03:51:11.856406 29477 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b762e765f74422885e822e73f8ef9e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:11 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:47738
I20260808 03:51:11.868438 29716 tablet_replica.cc:330] T 4b762e765f74422885e822e73f8ef9e1 P bd75c3ce1a174b62ae0bf9deef1e17ce: stopping tablet replica
I20260808 03:51:11.868789 29716 raft_consensus.cc:2229] T 4b762e765f74422885e822e73f8ef9e1 P bd75c3ce1a174b62ae0bf9deef1e17ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:51:11.869133 29716 raft_consensus.cc:2258] T 4b762e765f74422885e822e73f8ef9e1 P bd75c3ce1a174b62ae0bf9deef1e17ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:11.870512 29716 ts_tablet_manager.cc:1905] T 4b762e765f74422885e822e73f8ef9e1 P bd75c3ce1a174b62ae0bf9deef1e17ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:11.874212 29719 tablet_replica.cc:330] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d: stopping tablet replica
I20260808 03:51:11.874550 29719 raft_consensus.cc:2229] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:51:11.875386 29719 raft_consensus.cc:2258] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:11.876521 29719 ts_tablet_manager.cc:1905] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:11.880270 29717 tablet_replica.cc:330] T 4b762e765f74422885e822e73f8ef9e1 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: stopping tablet replica
I20260808 03:51:11.880507 29717 raft_consensus.cc:2229] T 4b762e765f74422885e822e73f8ef9e1 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:51:11.881019 29717 raft_consensus.cc:2258] T 4b762e765f74422885e822e73f8ef9e1 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:11.882198 29717 ts_tablet_manager.cc:1905] T 4b762e765f74422885e822e73f8ef9e1 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:11.883529 29719 ts_tablet_manager.cc:1918] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 03:51:11.883736 29719 log.cc:1196] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-0/wal/wals/4b762e765f74422885e822e73f8ef9e1
I20260808 03:51:11.884261 29719 ts_tablet_manager.cc:1939] T 4b762e765f74422885e822e73f8ef9e1 P abb3892fcf19418fa4b3a8b6328bbf6d: Deleting consensus metadata
Aug 08 03:51:11 dist-test-slave-jlzn krb5kdc[28810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161070, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.146.129@KRBTEST.COM
I20260808 03:51:11.885538 28853 catalog_manager.cc:4841] TS abb3892fcf19418fa4b3a8b6328bbf6d (127.26.146.129:43111): tablet 4b762e765f74422885e822e73f8ef9e1 (table test-table [id=8e9f1eaa112d481c9e8a5c1145302bb6]) successfully deleted
I20260808 03:51:11.890170 29716 ts_tablet_manager.cc:1918] T 4b762e765f74422885e822e73f8ef9e1 P bd75c3ce1a174b62ae0bf9deef1e17ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 03:51:11.890336 29716 log.cc:1196] T 4b762e765f74422885e822e73f8ef9e1 P bd75c3ce1a174b62ae0bf9deef1e17ce: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/4b762e765f74422885e822e73f8ef9e1
I20260808 03:51:11.891052 29716 ts_tablet_manager.cc:1939] T 4b762e765f74422885e822e73f8ef9e1 P bd75c3ce1a174b62ae0bf9deef1e17ce: Deleting consensus metadata
I20260808 03:51:11.893724 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.867063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54206 (local address 127.26.146.129:43111)
0808 03:51:11.867255 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:11.867262 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:11.873102 (+ 5840us) server_negotiation.cc:407] Connection header received
0808 03:51:11.873599 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:11.873606 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:11.873688 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:11.873770 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:11.875997 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.877119 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.879102 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.880105 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.880232 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.886701 (+ 6469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:11.886713 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:11.886727 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:11.886765 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:11.890060 (+ 3295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:11.890890 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:11.890898 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:11.890900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:11.891031 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:11.891456 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:11.891460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:11.891462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:11.892036 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:11.892419 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:11.893091 (+ 672us) server_negotiation.cc:299] Negotiation successful
0808 03:51:11.893362 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260808 03:51:11.895761 29167 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54206
I20260808 03:51:11.897114 29717 ts_tablet_manager.cc:1918] T 4b762e765f74422885e822e73f8ef9e1 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 03:51:11.898509 28848 catalog_manager.cc:4841] TS bd75c3ce1a174b62ae0bf9deef1e17ce (127.26.146.130:40697): tablet 4b762e765f74422885e822e73f8ef9e1 (table test-table [id=8e9f1eaa112d481c9e8a5c1145302bb6]) successfully deleted
I20260808 03:51:11.899081 29717 log.cc:1196] T 4b762e765f74422885e822e73f8ef9e1 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/4b762e765f74422885e822e73f8ef9e1
I20260808 03:51:11.899778 29717 ts_tablet_manager.cc:1939] T 4b762e765f74422885e822e73f8ef9e1 P 0dcf84aa6b2f4b9b8d7993ba1eb8d33e: Deleting consensus metadata
I20260808 03:51:11.901136 28854 catalog_manager.cc:4841] TS 0dcf84aa6b2f4b9b8d7993ba1eb8d33e (127.26.146.131:33885): tablet 4b762e765f74422885e822e73f8ef9e1 (table test-table [id=8e9f1eaa112d481c9e8a5c1145302bb6]) successfully deleted
I20260808 03:51:11.916462 29669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.901915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50358 (local address 127.26.146.190:40575)
0808 03:51:11.902170 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:11.902175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:11.902794 (+ 619us) server_negotiation.cc:407] Connection header received
0808 03:51:11.902942 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:11.902947 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:11.903012 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:11.903090 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:11.904890 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.905694 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.906975 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.907545 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.907662 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.910520 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:11.910525 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:11.910528 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:11.910562 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:11.913450 (+ 2888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:11.914151 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:11.914156 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:11.914158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:11.914314 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:11.914913 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:11.914918 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:11.914920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:11.915290 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:11.915545 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:11.915900 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 03:51:11.916148 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
W20260808 03:51:11.917375 28865 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50358
I20260808 03:51:11.919534 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 28974
I20260808 03:51:11.942670 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.921373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54208 (local address 127.26.146.129:43111)
0808 03:51:11.921585 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:11.921592 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:11.930108 (+ 8516us) server_negotiation.cc:407] Connection header received
0808 03:51:11.930284 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:11.930290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:11.930392 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:11.930508 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:11.931633 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.932509 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.933712 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.934154 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.934265 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.936605 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:11.936610 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:11.936613 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:11.936646 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:11.939642 (+ 2996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:11.940306 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:11.940310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:11.940313 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:11.940593 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:11.941054 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:11.941059 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:11.941063 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:11.941438 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:11.941855 (+ 417us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:11.942020 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 03:51:11.942261 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 03:51:11.943338 29167 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54208
W20260808 03:51:11.943948 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 28974: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:51:11.944216 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 28974
I20260808 03:51:11.969687 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 29178
Aug 08 03:51:11 dist-test-slave-jlzn krb5kdc[28810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161070, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.146.130@KRBTEST.COM
I20260808 03:51:11.985239 29601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:11.970413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48204 (local address 127.26.146.130:40697)
0808 03:51:11.970740 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:11.970747 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:11.970767 (+ 20us) server_negotiation.cc:407] Connection header received
0808 03:51:11.970878 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:11.970883 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:11.970949 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:11.971038 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:11.972249 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.973002 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.974236 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:11.974652 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:11.974769 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:11.978875 (+ 4106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:11.978887 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:11.978902 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:11.978939 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:11.982160 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:11.982810 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:11.982818 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:11.982821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:11.982976 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:11.983344 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:11.983348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:11.983350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:11.983722 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:11.984085 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:11.984586 (+ 501us) server_negotiation.cc:299] Negotiation successful
0808 03:51:11.984806 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168}
W20260808 03:51:11.985962 29366 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48204
W20260808 03:51:11.986742 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 29178: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:51:11.986995 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 29178
I20260808 03:51:12.022855 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 29381
Aug 08 03:51:12 dist-test-slave-jlzn krb5kdc[28810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161070, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.146.131@KRBTEST.COM
I20260808 03:51:12.043860 29603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:12.023790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51712 (local address 127.26.146.131:33885)
0808 03:51:12.024046 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:12.024053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:12.024400 (+ 347us) server_negotiation.cc:407] Connection header received
0808 03:51:12.024579 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:12.024585 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:12.024695 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:12.024806 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:12.026556 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:12.029884 (+ 3328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:12.031537 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:12.032290 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:12.032392 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:12.037775 (+ 5383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:12.037786 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:12.037802 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:12.037841 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:12.040905 (+ 3064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:12.041786 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:12.041793 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:12.041795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:12.041950 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:12.042348 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:12.042352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:12.042354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:12.042750 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:12.042985 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:12.043349 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 03:51:12.043512 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260808 03:51:12.044521 29571 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51712
W20260808 03:51:12.045105 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 29381: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:51:12.045300 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 29381
I20260808 03:51:12.069182 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 28829
I20260808 03:51:12.070330 28934 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:12.184461 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 28829
2026-08-08T03:51:12Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5187 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:51:12 dist-test-slave-jlzn krb5kdc[29731](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:51:12 dist-test-slave-jlzn krb5kdc[29731](info): set up 2 sockets
Aug 08 03:51:12 dist-test-slave-jlzn krb5kdc[29731](info): commencing operation
krb5kdc: starting...
W20260808 03:51:13.363319 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.001s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:51:13 dist-test-slave-jlzn krb5kdc[29731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:51:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:51:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:51:13.554595 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:44399
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:37759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:44399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:13.873597 29749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:13.873876 29749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:13.874002 29749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:13.881168 29749 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:51:13.881268 29749 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 03:51:13.881330 29749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:13.881410 29749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:13.881474 29749 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:51:13.881563 29749 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:51:13.891076 29749 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:37759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:44399
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:44399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:13.892344 29749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:13.893627 29749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:13.904594 29757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:13.904733 29762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:13.904781 29758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:13.906026 29749 server_base.cc:1034] running on GCE node
I20260808 03:51:13.906736 29749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:13.908043 29749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:13.909281 29749 hybrid_clock.cc:648] HybridClock initialized: now 1786161073909196 us; error 47 us; skew 500 ppm
I20260808 03:51:13.909584 29749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:13 dist-test-slave-jlzn krb5kdc[29731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161073, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:13.912726 29749 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:51:13.914028 29749 webserver.cc:460] Webserver started at http://127.26.146.190:32887/ using document root <none> and password file <none>
I20260808 03:51:13.914705 29749 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:13.914815 29749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:13.915110 29749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:13.917985 29749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "02e199820a3c4e39b4da67726453221e"
format_stamp: "Formatted at 2026-08-08 03:51:13 on dist-test-slave-jlzn"
server_key: "b3406038051fd2beb4108043d59a0a31"
server_key_iv: "990bb35c402d267e50113eb2a40adfb8"
server_key_version: "encryptionkey@0"
I20260808 03:51:13.918751 29749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "02e199820a3c4e39b4da67726453221e"
format_stamp: "Formatted at 2026-08-08 03:51:13 on dist-test-slave-jlzn"
server_key: "b3406038051fd2beb4108043d59a0a31"
server_key_iv: "990bb35c402d267e50113eb2a40adfb8"
server_key_version: "encryptionkey@0"
I20260808 03:51:13.923756 29749 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 03:51:13.927119 29773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:13.928386 29749 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:51:13.928537 29749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "02e199820a3c4e39b4da67726453221e"
format_stamp: "Formatted at 2026-08-08 03:51:13 on dist-test-slave-jlzn"
server_key: "b3406038051fd2beb4108043d59a0a31"
server_key_iv: "990bb35c402d267e50113eb2a40adfb8"
server_key_version: "encryptionkey@0"
I20260808 03:51:13.928748 29749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:13.946609 29749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:13.950390 29749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:13.950659 29749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:13.970875 29749 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:44399
I20260808 03:51:13.970892 29859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:44399 every 8 connection(s)
I20260808 03:51:13.972239 29749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:51:13.975854 29860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:13.980706 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 29749
I20260808 03:51:13.980902 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:51:13.981521 27210 external_mini_cluster.cc:1428] Setting key 996a4a122f35f8949e3aaa69ffb0201b
I20260808 03:51:13.986379 29860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e: Bootstrap starting.
I20260808 03:51:13.989321 29860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:13.990281 29860 log.cc:824] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:13.992995 29860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e: No bootstrap required, opened a new log
Aug 08 03:51:13 dist-test-slave-jlzn krb5kdc[29731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:14.000458 29860 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e199820a3c4e39b4da67726453221e" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44399 } }
I20260808 03:51:14.000802 29860 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:14.000911 29860 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02e199820a3c4e39b4da67726453221e, State: Initialized, Role: FOLLOWER
I20260808 03:51:14.001541 29860 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e199820a3c4e39b4da67726453221e" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44399 } }
I20260808 03:51:14.001785 29860 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:51:14.001911 29860 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:51:14.002077 29860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:14.002434 29867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:13.983464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47262 (local address 127.26.146.190:44399)
0808 03:51:13.984289 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:13.984303 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:13.984342 (+ 39us) server_negotiation.cc:407] Connection header received
0808 03:51:13.985998 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:13.986021 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:13.986380 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:13.986738 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:13.988206 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:13.989474 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:13.990722 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:13.991413 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:13.991626 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:13.995414 (+ 3788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:13.995422 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:13.995438 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:13.995480 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:13.998449 (+ 2969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:13.999107 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:13.999114 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:13.999119 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:13.999288 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:13.999655 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:13.999659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:13.999661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:14.000227 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:14.000529 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:14.000938 (+ 409us) server_negotiation.cc:299] Negotiation successful
0808 03:51:14.001381 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":379,"threads_started":1}
I20260808 03:51:14.004448 29860 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e199820a3c4e39b4da67726453221e" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44399 } }
I20260808 03:51:14.004922 29860 leader_election.cc:302] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02e199820a3c4e39b4da67726453221e; no voters:
I20260808 03:51:14.005517 29860 leader_election.cc:288] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:51:14.005635 29869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:14.006295 29869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e [term 1 LEADER]: Becoming Leader. State: Replica: 02e199820a3c4e39b4da67726453221e, State: Running, Role: LEADER
I20260808 03:51:14.006889 29869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e199820a3c4e39b4da67726453221e" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44399 } }
I20260808 03:51:14.007524 29860 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:51:14.009871 29871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02e199820a3c4e39b4da67726453221e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e199820a3c4e39b4da67726453221e" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44399 } } }
I20260808 03:51:14.009903 29872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02e199820a3c4e39b4da67726453221e. Latest consensus state: current_term: 1 leader_uuid: "02e199820a3c4e39b4da67726453221e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e199820a3c4e39b4da67726453221e" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44399 } } }
I20260808 03:51:14.010250 29871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:14.010254 29872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:14.016809 29880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:51:14.021365 29880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:51:14.029065 29880 catalog_manager.cc:1347] Generated new cluster ID: 83b6ec5a8106467ca3030f387f85c30a
I20260808 03:51:14.029172 29880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:51:14.040843 29880 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:51:14.042554 29880 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:51:14.050604 29880 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e: Generated new TSK 0
I20260808 03:51:14.051575 29880 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
I20260808 03:51:14.100746 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:44399
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:37759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:14.422127 29903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:14.422394 29903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:14.422533 29903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:14.429505 29903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:14.429607 29903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:14.429724 29903 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 03:51:14.429786 29903 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 03:51:14.429875 29903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:51:14.439327 29903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:37759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.146.190:44399
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:14.440526 29903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:14.441811 29903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:14.453970 29919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:14.453941 29917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:14.454264 29913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:14.454664 29903 server_base.cc:1034] running on GCE node
I20260808 03:51:14.455255 29903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:14.456243 29903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:14.457502 29903 hybrid_clock.cc:648] HybridClock initialized: now 1786161074457440 us; error 40 us; skew 500 ppm
I20260808 03:51:14.457813 29903 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:14 dist-test-slave-jlzn krb5kdc[29731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161074, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:14.460870 29903 init.cc:377] Logged in from keytab as kudu/127.26.146.129@KRBTEST.COM (short username kudu)
I20260808 03:51:14.462204 29903 webserver.cc:460] Webserver started at http://127.26.146.129:43823/ using document root <none> and password file <none>
I20260808 03:51:14.462813 29903 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:14.462914 29903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:14.463184 29903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:14.465998 29903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f"
format_stamp: "Formatted at 2026-08-08 03:51:14 on dist-test-slave-jlzn"
server_key: "02878e98bf3ed33d00e5b123f77469f4"
server_key_iv: "7c6147cc8100911fd371adfcbbf32196"
server_key_version: "encryptionkey@0"
I20260808 03:51:14.466691 29903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f"
format_stamp: "Formatted at 2026-08-08 03:51:14 on dist-test-slave-jlzn"
server_key: "02878e98bf3ed33d00e5b123f77469f4"
server_key_iv: "7c6147cc8100911fd371adfcbbf32196"
server_key_version: "encryptionkey@0"
I20260808 03:51:14.471702 29903 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260808 03:51:14.474968 29926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:14.476261 29903 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 03:51:14.476403 29903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f"
format_stamp: "Formatted at 2026-08-08 03:51:14 on dist-test-slave-jlzn"
server_key: "02878e98bf3ed33d00e5b123f77469f4"
server_key_iv: "7c6147cc8100911fd371adfcbbf32196"
server_key_version: "encryptionkey@0"
I20260808 03:51:14.476572 29903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:14.493973 29903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:14.497992 29903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:14.498265 29903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:14.500813 29903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:14.500916 29903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:14.501021 29903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:14.501108 29903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 03:51:14 dist-test-slave-jlzn krb5kdc[29731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161074, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:14.524588 29867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:14.501314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:60593 (local address 127.26.146.190:44399)
0808 03:51:14.501520 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:14.501527 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:14.502905 (+ 1378us) server_negotiation.cc:407] Connection header received
0808 03:51:14.505018 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:14.505026 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:14.505123 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:14.505239 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:14.506916 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:14.508081 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:14.510647 (+ 2566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:14.511415 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:14.511558 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:14.515845 (+ 4287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:14.515853 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:14.515859 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:14.515903 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:14.519022 (+ 3119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:14.520053 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:14.520059 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:14.520062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:14.520217 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:14.520890 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:14.520895 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:14.520898 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:14.521173 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:14.521290 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:14.524021 (+ 2731us) server_negotiation.cc:299] Negotiation successful
0808 03:51:14.524248 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 03:51:14.524678 29938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:14.501874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:44399 (local address 127.26.146.129:60593)
0808 03:51:14.502735 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:14.502779 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:14.504744 (+ 1965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:14.505493 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:14.505505 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:14.506205 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:14.506741 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:14.506758 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:14.508272 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:14.508278 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:14.510493 (+ 2215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:14.510510 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:14.511631 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:14.511636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:14.511860 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:14.512559 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:14.512593 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:14.515581 (+ 2988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:14.519278 (+ 3697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:14.519288 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:14.519302 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:14.519926 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:14.520403 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:14.520408 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:14.520415 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:14.520739 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:14.521305 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:14.521314 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:14.521841 (+ 527us) client_negotiation.cc:769] Sending connection context
0808 03:51:14.522243 (+ 402us) client_negotiation.cc:240] Negotiation successful
0808 03:51:14.522849 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":370,"threads_started":1}
I20260808 03:51:14.545025 29903 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:37659
I20260808 03:51:14.545039 30105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:37659 every 8 connection(s)
I20260808 03:51:14.545532 30061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:14.531642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:44399 (local address 127.26.146.129:40871)
0808 03:51:14.532287 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:14.532303 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:14.532406 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:14.532707 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:14.532711 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:14.533171 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:14.533459 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:14.533468 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:14.534893 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:14.534897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:14.535928 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:14.535939 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:14.537023 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:14.537031 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:14.537182 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:14.537721 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:14.537741 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:14.538714 (+ 973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:14.542441 (+ 3727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:14.542447 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:14.542450 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:14.543082 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:14.543534 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:14.543538 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:14.543540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:14.543661 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:14.544320 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:14.544324 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:14.544548 (+ 224us) client_negotiation.cc:769] Sending connection context
0808 03:51:14.544695 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 03:51:14.545072 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":366,"threads_started":1}
I20260808 03:51:14.546607 29903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:51:14.546618 29867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:14.531552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:40871 (local address 127.26.146.190:44399)
0808 03:51:14.531700 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:14.531704 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:14.532395 (+ 691us) server_negotiation.cc:407] Connection header received
0808 03:51:14.532496 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:14.532502 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:14.532584 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:14.532689 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:14.533585 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:14.534767 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:14.536068 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:14.536653 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:14.536769 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:14.538899 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:14.538905 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:14.538910 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:14.538948 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:14.542265 (+ 3317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:14.543266 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:14.543271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:14.543274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:14.543425 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:14.543772 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:14.543777 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:14.543781 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:14.544159 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:14.545920 (+ 1761us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:14.546070 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 03:51:14.546291 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 03:51:14.549254 30106 heartbeater.cc:347] Connected to a master server at 127.26.146.190:44399
I20260808 03:51:14.549480 30106 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:14.550024 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 29903
I20260808 03:51:14.550282 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:51:14.550292 30106 heartbeater.cc:511] Master 127.26.146.190:44399 requested a full tablet report, sending...
I20260808 03:51:14.551097 27210 external_mini_cluster.cc:1428] Setting key 28ada4b29514f9172acf9b09dd5e43de
I20260808 03:51:14.552426 29797 ts_manager.cc:194] Registered new tserver with Master: 6ae07d49ebd941b2bf8e3fde33a76b6f (127.26.146.129:37659)
I20260808 03:51:14.554153 29797 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.129@KRBTEST.COM'} at 127.26.146.129:60593
WARNING: no policy specified for kudu/127.26.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
I20260808 03:51:14.595268 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:44399
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:37759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:14.912587 30112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:14.912869 30112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:14.913008 30112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:14.920526 30112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:14.920635 30112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:14.920769 30112 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 03:51:14.920832 30112 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 03:51:14.920931 30112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:51:14.930752 30112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:37759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.146.190:44399
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:14.932156 30112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:14.933557 30112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:14.945989 30123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:14.946046 30126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:14.946165 30128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:14.946875 30112 server_base.cc:1034] running on GCE node
I20260808 03:51:14.947464 30112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:14.948449 30112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:14.949679 30112 hybrid_clock.cc:648] HybridClock initialized: now 1786161074949623 us; error 39 us; skew 500 ppm
I20260808 03:51:14.949980 30112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:14 dist-test-slave-jlzn krb5kdc[29731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161074, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:14.956003 30112 init.cc:377] Logged in from keytab as kudu/127.26.146.130@KRBTEST.COM (short username kudu)
I20260808 03:51:14.957391 30112 webserver.cc:460] Webserver started at http://127.26.146.130:38825/ using document root <none> and password file <none>
I20260808 03:51:14.958016 30112 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:14.958117 30112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:14.958384 30112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:14.961220 30112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "cae774377c4242fb87db8fa139f33c55"
format_stamp: "Formatted at 2026-08-08 03:51:14 on dist-test-slave-jlzn"
server_key: "cdd0b40ce19d89f73c28ae5e68e3e027"
server_key_iv: "ceb8974340dfc4781442a71bb74144a3"
server_key_version: "encryptionkey@0"
I20260808 03:51:14.961927 30112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "cae774377c4242fb87db8fa139f33c55"
format_stamp: "Formatted at 2026-08-08 03:51:14 on dist-test-slave-jlzn"
server_key: "cdd0b40ce19d89f73c28ae5e68e3e027"
server_key_iv: "ceb8974340dfc4781442a71bb74144a3"
server_key_version: "encryptionkey@0"
I20260808 03:51:14.966784 30112 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260808 03:51:14.970088 30138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:14.971429 30112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 03:51:14.971586 30112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "cae774377c4242fb87db8fa139f33c55"
format_stamp: "Formatted at 2026-08-08 03:51:14 on dist-test-slave-jlzn"
server_key: "cdd0b40ce19d89f73c28ae5e68e3e027"
server_key_iv: "ceb8974340dfc4781442a71bb74144a3"
server_key_version: "encryptionkey@0"
I20260808 03:51:14.971776 30112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:15.003147 30112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:15.007025 30112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:15.007313 30112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:15.010010 30112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:15.010107 30112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:15.010226 30112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:15.010311 30112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 03:51:15 dist-test-slave-jlzn krb5kdc[29731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161074, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:15.031610 29867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:15.010510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:55573 (local address 127.26.146.190:44399)
0808 03:51:15.010737 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:15.010744 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:15.012232 (+ 1488us) server_negotiation.cc:407] Connection header received
0808 03:51:15.014444 (+ 2212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:15.014451 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:15.014545 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:15.014662 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:15.016748 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.017636 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.020150 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.020578 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.020709 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:15.025201 (+ 4492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:15.025207 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:15.025211 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:15.025250 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:15.028504 (+ 3254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:15.029181 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:15.029187 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:15.029191 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:15.029349 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:15.029919 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:15.029925 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:15.029928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:15.030213 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:15.030346 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:15.031032 (+ 686us) server_negotiation.cc:299] Negotiation successful
0808 03:51:15.031273 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 03:51:15.032716 30151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:15.011146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:44399 (local address 127.26.146.130:55573)
0808 03:51:15.012021 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:15.012093 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:15.014084 (+ 1991us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:15.014949 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:15.014959 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:15.015645 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:15.016523 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:15.016549 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.017836 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.017841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:15.019996 (+ 2155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:15.020008 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.020848 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.020857 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:15.021161 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:15.021832 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:15.021861 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:15.024970 (+ 3109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:15.028635 (+ 3665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:15.028647 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:15.028662 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:15.029056 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:15.029496 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:15.029500 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:15.029503 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:15.029756 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:15.030336 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:15.030343 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:15.030679 (+ 336us) client_negotiation.cc:769] Sending connection context
0808 03:51:15.030904 (+ 225us) client_negotiation.cc:240] Negotiation successful
0808 03:51:15.031427 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":356,"threads_started":1}
I20260808 03:51:15.053244 30112 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:33031
I20260808 03:51:15.053267 30322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:33031 every 8 connection(s)
I20260808 03:51:15.054021 29867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:15.039381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:41589 (local address 127.26.146.190:44399)
0808 03:51:15.039533 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:15.039537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:15.040244 (+ 707us) server_negotiation.cc:407] Connection header received
0808 03:51:15.040413 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:15.040418 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:15.040479 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:15.040554 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:15.041911 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.042791 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.044761 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.045288 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.045393 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:15.047834 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:15.047842 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:15.047847 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:15.047884 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:15.051315 (+ 3431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:15.051970 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:15.051977 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:15.051981 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:15.052131 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:15.052545 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:15.052550 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:15.052553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:15.052878 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:15.053011 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:15.053458 (+ 447us) server_negotiation.cc:299] Negotiation successful
0808 03:51:15.053674 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 03:51:15.054060 30262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:15.039497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:44399 (local address 127.26.146.130:41589)
0808 03:51:15.040144 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:15.040163 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:15.040298 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:15.040600 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:15.040604 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:15.041308 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:15.041781 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:15.041794 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.042962 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.042967 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:15.044575 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:15.044592 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.045475 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.045485 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:15.045646 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:15.046222 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:15.046249 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:15.047547 (+ 1298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:15.051441 (+ 3894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:15.051445 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:15.051448 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:15.051861 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:15.052260 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:15.052265 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:15.052268 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:15.052423 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:15.052993 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:15.052998 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:15.053190 (+ 192us) client_negotiation.cc:769] Sending connection context
0808 03:51:15.053309 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 03:51:15.053620 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":312,"threads_started":1}
I20260808 03:51:15.054704 30112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:51:15.056770 30323 heartbeater.cc:347] Connected to a master server at 127.26.146.190:44399
I20260808 03:51:15.056973 30323 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:15.057590 30323 heartbeater.cc:511] Master 127.26.146.190:44399 requested a full tablet report, sending...
I20260808 03:51:15.058748 29797 ts_manager.cc:194] Registered new tserver with Master: cae774377c4242fb87db8fa139f33c55 (127.26.146.130:33031)
I20260808 03:51:15.059535 29797 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.130@KRBTEST.COM'} at 127.26.146.130:55573
I20260808 03:51:15.061973 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 30112
I20260808 03:51:15.062134 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:51:15.062529 27210 external_mini_cluster.cc:1428] Setting key e7fa9e26cbb7a3dd1602847442c9ca0d
WARNING: no policy specified for kudu/127.26.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
I20260808 03:51:15.107335 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:44399
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:37759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:15.426632 30331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:15.426909 30331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:15.427057 30331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:15.434245 30331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:15.434363 30331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:15.434495 30331 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 03:51:15.434573 30331 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 03:51:15.434679 30331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:51:15.444298 30331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:37759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.146.190:44399
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:15.445619 30331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:15.446929 30331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:15.459314 30339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:15.459362 30340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:15.459398 30342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:15.460345 30331 server_base.cc:1034] running on GCE node
I20260808 03:51:15.460933 30331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:15.461791 30331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:15.462998 30331 hybrid_clock.cc:648] HybridClock initialized: now 1786161075462954 us; error 31 us; skew 500 ppm
I20260808 03:51:15.463305 30331 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:15 dist-test-slave-jlzn krb5kdc[29731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161075, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:15.466359 30331 init.cc:377] Logged in from keytab as kudu/127.26.146.131@KRBTEST.COM (short username kudu)
I20260808 03:51:15.467715 30331 webserver.cc:460] Webserver started at http://127.26.146.131:39715/ using document root <none> and password file <none>
I20260808 03:51:15.468356 30331 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:15.468456 30331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:15.468719 30331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:15.471400 30331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "5c3f01dda9234f2aa081ad4435bf94a1"
format_stamp: "Formatted at 2026-08-08 03:51:15 on dist-test-slave-jlzn"
server_key: "88e63c32f6b95310efb8bc16e1abd34f"
server_key_iv: "58537dbe764d5d7cd81cace3fa6c1193"
server_key_version: "encryptionkey@0"
I20260808 03:51:15.472153 30331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "5c3f01dda9234f2aa081ad4435bf94a1"
format_stamp: "Formatted at 2026-08-08 03:51:15 on dist-test-slave-jlzn"
server_key: "88e63c32f6b95310efb8bc16e1abd34f"
server_key_iv: "58537dbe764d5d7cd81cace3fa6c1193"
server_key_version: "encryptionkey@0"
I20260808 03:51:15.476981 30331 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 03:51:15.480378 30352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:15.481714 30331 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:51:15.481863 30331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "5c3f01dda9234f2aa081ad4435bf94a1"
format_stamp: "Formatted at 2026-08-08 03:51:15 on dist-test-slave-jlzn"
server_key: "88e63c32f6b95310efb8bc16e1abd34f"
server_key_iv: "58537dbe764d5d7cd81cace3fa6c1193"
server_key_version: "encryptionkey@0"
I20260808 03:51:15.482033 30331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:15.501646 30331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:15.505415 30331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:15.505679 30331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:15.508355 30331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:15.508474 30331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:15.508656 30331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:15.508777 30331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 03:51:15 dist-test-slave-jlzn krb5kdc[29731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161075, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:15.531466 30363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:15.509456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:44399 (local address 127.26.146.131:55281)
0808 03:51:15.510228 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:15.510271 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:15.512069 (+ 1798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:15.512838 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:15.512849 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:15.513681 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:15.514556 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:15.514585 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.515970 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.515976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:15.518164 (+ 2188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:15.518179 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.519060 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.519068 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:15.519363 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:15.520125 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:15.520163 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:15.523122 (+ 2959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:15.526549 (+ 3427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:15.526558 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:15.526576 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:15.527257 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:15.527703 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:15.527711 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:15.527715 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:15.528031 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:15.528643 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:15.528651 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:15.529122 (+ 471us) client_negotiation.cc:769] Sending connection context
0808 03:51:15.529419 (+ 297us) client_negotiation.cc:240] Negotiation successful
0808 03:51:15.529943 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":349,"threads_started":1}
I20260808 03:51:15.532495 29867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:15.508798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:55281 (local address 127.26.146.190:44399)
0808 03:51:15.508968 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:15.508973 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:15.510385 (+ 1412us) server_negotiation.cc:407] Connection header received
0808 03:51:15.512350 (+ 1965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:15.512356 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:15.512443 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:15.512530 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:15.514804 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.515798 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.518310 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.518771 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.518898 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:15.523340 (+ 4442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:15.523347 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:15.523351 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:15.523383 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:15.526424 (+ 3041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:15.527424 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:15.527430 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:15.527433 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:15.527593 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:15.528158 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:15.528163 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:15.528165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:15.528529 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:15.528673 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:15.532024 (+ 3351us) server_negotiation.cc:299] Negotiation successful
0808 03:51:15.532201 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 03:51:15.556852 30106 heartbeater.cc:503] Master 127.26.146.190:44399 was elected leader, sending a full tablet report...
I20260808 03:51:15.560631 29867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:15.546635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:54589 (local address 127.26.146.190:44399)
0808 03:51:15.546827 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:15.546834 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:15.547458 (+ 624us) server_negotiation.cc:407] Connection header received
0808 03:51:15.547772 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:15.547777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:15.547897 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:15.547971 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:15.549489 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.550661 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.552001 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.552565 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.552707 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:15.554771 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:15.554777 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:15.554782 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:15.554813 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:15.557856 (+ 3043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:15.558580 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:15.558586 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:15.558590 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:15.558763 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:15.559126 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:15.559131 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:15.559135 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:15.559520 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:15.559694 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:15.560067 (+ 373us) server_negotiation.cc:299] Negotiation successful
0808 03:51:15.560264 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 03:51:15.560724 30479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:15.546711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:44399 (local address 127.26.146.131:54589)
0808 03:51:15.547345 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:15.547365 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:15.547587 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:15.548192 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:15.548196 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:15.548898 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:15.549255 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:15.549268 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.550808 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.550812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:15.551856 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:15.551870 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.552762 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.552770 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:15.552936 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:15.553600 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:15.553627 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:15.554603 (+ 976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:15.558021 (+ 3418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:15.558026 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:15.558029 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:15.558469 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:15.558898 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:15.558903 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:15.558907 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:15.559037 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:15.559622 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:15.559624 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:15.559773 (+ 149us) client_negotiation.cc:769] Sending connection context
0808 03:51:15.559956 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 03:51:15.560291 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":332,"threads_started":1}
I20260808 03:51:15.567186 30331 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:36241
I20260808 03:51:15.567224 30519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:36241 every 8 connection(s)
I20260808 03:51:15.568372 30331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:51:15.570190 30520 heartbeater.cc:347] Connected to a master server at 127.26.146.190:44399
I20260808 03:51:15.570384 30520 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:15.570937 30520 heartbeater.cc:511] Master 127.26.146.190:44399 requested a full tablet report, sending...
I20260808 03:51:15.572194 29797 ts_manager.cc:194] Registered new tserver with Master: 5c3f01dda9234f2aa081ad4435bf94a1 (127.26.146.131:36241)
I20260808 03:51:15.573139 29797 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.131@KRBTEST.COM'} at 127.26.146.131:55281
I20260808 03:51:15.575537 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 30331
I20260808 03:51:15.575716 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:51:15.576176 27210 external_mini_cluster.cc:1428] Setting key a2cc1618dc93793ac592963ccb81f965
I20260808 03:51:15.578887 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 03:51:15 dist-test-slave-jlzn krb5kdc[29731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 03:51:15 dist-test-slave-jlzn krb5kdc[29731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:15.628644 29867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:15.615255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47264 (local address 127.26.146.190:44399)
0808 03:51:15.615481 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:15.615487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:15.615669 (+ 182us) server_negotiation.cc:407] Connection header received
0808 03:51:15.615921 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:15.615927 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:15.616019 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:15.616108 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:15.617547 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.618295 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.619514 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.620005 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.620114 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:15.623765 (+ 3651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:15.623770 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:15.623774 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:15.623842 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:15.626117 (+ 2275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:15.626715 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:15.626719 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:15.626721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:15.626845 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:15.627181 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:15.627184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:15.627186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:15.627482 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:15.627721 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:15.628108 (+ 387us) server_negotiation.cc:299] Negotiation successful
0808 03:51:15.628335 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 03:51:15.632479 29797 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47264:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:51:15.639456 29797 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:51:15.671057 30542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:15.650456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:37659 (local address 127.0.0.1:46470)
0808 03:51:15.662063 (+ 11607us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:15.662098 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:15.662264 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:15.664108 (+ 1844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:15.664116 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:15.664159 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:15.664636 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:15.664649 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.666454 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.666460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:15.668335 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:15.668348 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.670000 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.670006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:15.670189 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:15.670251 (+ 62us) client_negotiation.cc:769] Sending connection context
0808 03:51:15.670383 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 03:51:15.670574 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11389,"mutex_wait_us":58,"spinlock_wait_cycles":896,"thread_start_us":13140,"threads_started":1}
I20260808 03:51:15.671193 30543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:15.651000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37352 (local address 127.26.146.130:33031)
0808 03:51:15.661403 (+ 10403us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:15.661412 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:15.663365 (+ 1953us) server_negotiation.cc:407] Connection header received
0808 03:51:15.663548 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:15.663557 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:15.663793 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:15.664000 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:15.664713 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.666188 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.668893 (+ 2705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.670186 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.670339 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:15.670498 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:15.670745 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 03:51:15.670913 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10229,"thread_start_us":10410,"threads_started":1}
I20260808 03:51:15.671603 30547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:15.650693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:33031 (local address 127.0.0.1:37352)
0808 03:51:15.663247 (+ 12554us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:15.663266 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:15.663401 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:15.664016 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:15.664024 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:15.664056 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:15.664517 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:15.664530 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.666365 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.666370 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:15.668712 (+ 2342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:15.668727 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.670384 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.670390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:15.670572 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:15.670595 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 03:51:15.670686 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 03:51:15.670805 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12401,"mutex_wait_us":27,"thread_start_us":11025,"threads_started":1}
I20260808 03:51:15.672029 30540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:15.652730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:36241 (local address 127.0.0.1:56228)
0808 03:51:15.663885 (+ 11155us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:15.663902 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:15.664010 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:15.664526 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:15.664530 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:15.664561 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:15.665024 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:15.665031 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.667024 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.667029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:15.669161 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:15.669175 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.671167 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.671173 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:15.671346 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:15.671376 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 03:51:15.671457 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 03:51:15.671602 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11022,"mutex_wait_us":18,"thread_start_us":10212,"threads_started":1}
I20260808 03:51:15.672201 30545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:15.652260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56228 (local address 127.26.146.131:36241)
0808 03:51:15.660868 (+ 8608us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:15.660878 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:15.663993 (+ 3115us) server_negotiation.cc:407] Connection header received
0808 03:51:15.664210 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:15.664215 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:15.664353 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:15.664519 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:15.665204 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.666870 (+ 1666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.669331 (+ 2461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.670665 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.670851 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:15.671012 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:15.671713 (+ 701us) server_negotiation.cc:299] Negotiation successful
0808 03:51:15.671896 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8403,"thread_start_us":8652,"threads_started":1}
I20260808 03:51:15.672322 30541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:15.650653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46470 (local address 127.26.146.129:37659)
0808 03:51:15.662838 (+ 12185us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:15.662847 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:15.662877 (+ 30us) server_negotiation.cc:407] Connection header received
0808 03:51:15.662986 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:15.662994 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:15.663239 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:15.664650 (+ 1411us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:15.664878 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.666283 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.668484 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.669761 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.669976 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:15.670163 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:15.671864 (+ 1701us) server_negotiation.cc:299] Negotiation successful
0808 03:51:15.672043 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12027,"thread_start_us":11882,"threads_started":1}
I20260808 03:51:15.675417 30425 tablet_service.cc:1467] Processing CreateTablet for tablet a540a95833884776b10317172bd0b1f9 (DEFAULT_TABLE table=test-table [id=ea02461482b84ad8b575f6628081895a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:15.675426 30216 tablet_service.cc:1467] Processing CreateTablet for tablet a540a95833884776b10317172bd0b1f9 (DEFAULT_TABLE table=test-table [id=ea02461482b84ad8b575f6628081895a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:15.676782 30425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a540a95833884776b10317172bd0b1f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:15.676832 30216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a540a95833884776b10317172bd0b1f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:15.676400 29995 tablet_service.cc:1467] Processing CreateTablet for tablet a540a95833884776b10317172bd0b1f9 (DEFAULT_TABLE table=test-table [id=ea02461482b84ad8b575f6628081895a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:15.677580 29995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a540a95833884776b10317172bd0b1f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:15.687901 30549 tablet_bootstrap.cc:492] T a540a95833884776b10317172bd0b1f9 P 5c3f01dda9234f2aa081ad4435bf94a1: Bootstrap starting.
I20260808 03:51:15.690356 30550 tablet_bootstrap.cc:492] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55: Bootstrap starting.
I20260808 03:51:15.691620 30549 tablet_bootstrap.cc:654] T a540a95833884776b10317172bd0b1f9 P 5c3f01dda9234f2aa081ad4435bf94a1: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:15.692399 30551 tablet_bootstrap.cc:492] T a540a95833884776b10317172bd0b1f9 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Bootstrap starting.
I20260808 03:51:15.693305 30549 log.cc:824] T a540a95833884776b10317172bd0b1f9 P 5c3f01dda9234f2aa081ad4435bf94a1: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:15.694133 30550 tablet_bootstrap.cc:654] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:15.695178 30551 tablet_bootstrap.cc:654] T a540a95833884776b10317172bd0b1f9 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:15.695165 30550 log.cc:824] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:15.696275 30551 log.cc:824] T a540a95833884776b10317172bd0b1f9 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:15.696327 30549 tablet_bootstrap.cc:492] T a540a95833884776b10317172bd0b1f9 P 5c3f01dda9234f2aa081ad4435bf94a1: No bootstrap required, opened a new log
I20260808 03:51:15.696578 30549 ts_tablet_manager.cc:1397] T a540a95833884776b10317172bd0b1f9 P 5c3f01dda9234f2aa081ad4435bf94a1: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260808 03:51:15.698041 30550 tablet_bootstrap.cc:492] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55: No bootstrap required, opened a new log
I20260808 03:51:15.698287 30550 ts_tablet_manager.cc:1397] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260808 03:51:15.704738 30551 tablet_bootstrap.cc:492] T a540a95833884776b10317172bd0b1f9 P 6ae07d49ebd941b2bf8e3fde33a76b6f: No bootstrap required, opened a new log
I20260808 03:51:15.704958 30551 ts_tablet_manager.cc:1397] T a540a95833884776b10317172bd0b1f9 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.000s
I20260808 03:51:15.706619 30549 raft_consensus.cc:348] T a540a95833884776b10317172bd0b1f9 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:15.707032 30549 raft_consensus.cc:374] T a540a95833884776b10317172bd0b1f9 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:15.707141 30549 raft_consensus.cc:729] T a540a95833884776b10317172bd0b1f9 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3f01dda9234f2aa081ad4435bf94a1, State: Initialized, Role: FOLLOWER
I20260808 03:51:15.707777 30549 consensus_queue.cc:260] T a540a95833884776b10317172bd0b1f9 P 5c3f01dda9234f2aa081ad4435bf94a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:15.708140 30550 raft_consensus.cc:348] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:15.708637 30550 raft_consensus.cc:374] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:15.708743 30550 raft_consensus.cc:729] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cae774377c4242fb87db8fa139f33c55, State: Initialized, Role: FOLLOWER
I20260808 03:51:15.709329 30550 consensus_queue.cc:260] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:15.711532 30323 heartbeater.cc:503] Master 127.26.146.190:44399 was elected leader, sending a full tablet report...
I20260808 03:51:15.712332 30550 ts_tablet_manager.cc:1428] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260808 03:51:15.713271 30551 raft_consensus.cc:348] T a540a95833884776b10317172bd0b1f9 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:15.713572 30551 raft_consensus.cc:374] T a540a95833884776b10317172bd0b1f9 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:15.713668 30551 raft_consensus.cc:729] T a540a95833884776b10317172bd0b1f9 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ae07d49ebd941b2bf8e3fde33a76b6f, State: Initialized, Role: FOLLOWER
I20260808 03:51:15.714159 30551 consensus_queue.cc:260] T a540a95833884776b10317172bd0b1f9 P 6ae07d49ebd941b2bf8e3fde33a76b6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:15.715727 30549 ts_tablet_manager.cc:1428] T a540a95833884776b10317172bd0b1f9 P 5c3f01dda9234f2aa081ad4435bf94a1: Time spent starting tablet: real 0.019s user 0.009s sys 0.009s
I20260808 03:51:15.716118 30520 heartbeater.cc:503] Master 127.26.146.190:44399 was elected leader, sending a full tablet report...
I20260808 03:51:15.716797 30551 ts_tablet_manager.cc:1428] T a540a95833884776b10317172bd0b1f9 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260808 03:51:15.729149 30559 raft_consensus.cc:482] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:51:15.729415 30559 raft_consensus.cc:504] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:15.730870 30559 leader_election.cc:288] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ae07d49ebd941b2bf8e3fde33a76b6f (127.26.146.129:37659), 5c3f01dda9234f2aa081ad4435bf94a1 (127.26.146.131:36241)
I20260808 03:51:15.737978 30545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:15.731322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:37141 (local address 127.26.146.131:36241)
0808 03:51:15.731526 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:15.731532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:15.732377 (+ 845us) server_negotiation.cc:407] Connection header received
0808 03:51:15.732613 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:15.732618 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:15.732685 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:15.732777 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:15.733361 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.734149 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.736214 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.737028 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.737163 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:15.737231 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:15.737599 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 03:51:15.737729 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 03:51:15.737982 30541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:15.731142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:55727 (local address 127.26.146.129:37659)
0808 03:51:15.731339 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:15.731345 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:15.732218 (+ 873us) server_negotiation.cc:407] Connection header received
0808 03:51:15.732406 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:15.732411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:15.732482 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:15.732561 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:15.733247 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.734090 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.736364 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.737120 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.737271 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:15.737341 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:15.737627 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 03:51:15.737761 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 03:51:15.738130 30561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:15.731309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:37659 (local address 127.26.146.130:55727)
0808 03:51:15.732110 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:15.732132 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:15.732268 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:15.732603 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:15.732608 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:15.732643 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:15.733063 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:15.733080 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.734234 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.734240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:15.736200 (+ 1960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:15.736215 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.737340 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.737345 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:15.737473 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:15.737496 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 03:51:15.737585 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 03:51:15.737727 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"mutex_wait_us":52,"thread_start_us":679,"threads_started":1}
I20260808 03:51:15.738317 30562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:15.731360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:36241 (local address 127.26.146.130:37141)
0808 03:51:15.732279 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:15.732293 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:15.732455 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:15.732803 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:15.732806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:15.732829 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:15.733228 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:15.733240 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.734296 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.734301 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:15.736062 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:15.736075 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.737262 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.737269 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:15.737435 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:15.737466 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 03:51:15.737537 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 03:51:15.737654 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"mutex_wait_us":37,"thread_start_us":360,"threads_started":1}
I20260808 03:51:15.739117 30030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a540a95833884776b10317172bd0b1f9" candidate_uuid: "cae774377c4242fb87db8fa139f33c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" is_pre_election: true
I20260808 03:51:15.739612 30030 raft_consensus.cc:2454] T a540a95833884776b10317172bd0b1f9 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cae774377c4242fb87db8fa139f33c55 in term 0.
I20260808 03:51:15.740269 30145 leader_election.cc:302] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ae07d49ebd941b2bf8e3fde33a76b6f, cae774377c4242fb87db8fa139f33c55; no voters:
I20260808 03:51:15.740639 30559 raft_consensus.cc:2781] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:51:15.740792 30559 raft_consensus.cc:482] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:51:15.740902 30559 raft_consensus.cc:3037] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:15.740316 30453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a540a95833884776b10317172bd0b1f9" candidate_uuid: "cae774377c4242fb87db8fa139f33c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" is_pre_election: true
I20260808 03:51:15.741930 30453 raft_consensus.cc:2454] T a540a95833884776b10317172bd0b1f9 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cae774377c4242fb87db8fa139f33c55 in term 0.
I20260808 03:51:15.743518 30559 raft_consensus.cc:504] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:15.744079 30559 leader_election.cc:288] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55 [CANDIDATE]: Term 1 election: Requested vote from peers 6ae07d49ebd941b2bf8e3fde33a76b6f (127.26.146.129:37659), 5c3f01dda9234f2aa081ad4435bf94a1 (127.26.146.131:36241)
I20260808 03:51:15.744470 30030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a540a95833884776b10317172bd0b1f9" candidate_uuid: "cae774377c4242fb87db8fa139f33c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f"
I20260808 03:51:15.744536 30453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a540a95833884776b10317172bd0b1f9" candidate_uuid: "cae774377c4242fb87db8fa139f33c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3f01dda9234f2aa081ad4435bf94a1"
I20260808 03:51:15.744626 30030 raft_consensus.cc:3037] T a540a95833884776b10317172bd0b1f9 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:15.744724 30453 raft_consensus.cc:3037] T a540a95833884776b10317172bd0b1f9 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:15.746699 30030 raft_consensus.cc:2454] T a540a95833884776b10317172bd0b1f9 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cae774377c4242fb87db8fa139f33c55 in term 1.
I20260808 03:51:15.746884 30453 raft_consensus.cc:2454] T a540a95833884776b10317172bd0b1f9 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cae774377c4242fb87db8fa139f33c55 in term 1.
I20260808 03:51:15.747043 30145 leader_election.cc:302] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ae07d49ebd941b2bf8e3fde33a76b6f, cae774377c4242fb87db8fa139f33c55; no voters:
I20260808 03:51:15.747350 30559 raft_consensus.cc:2781] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:15.747749 30559 raft_consensus.cc:686] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55 [term 1 LEADER]: Becoming Leader. State: Replica: cae774377c4242fb87db8fa139f33c55, State: Running, Role: LEADER
I20260808 03:51:15.748426 30559 consensus_queue.cc:237] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:15.752568 29794 catalog_manager.cc:5495] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55 reported cstate change: term changed from 0 to 1, leader changed from <none> to cae774377c4242fb87db8fa139f33c55 (127.26.146.130). New cstate: current_term: 1 leader_uuid: "cae774377c4242fb87db8fa139f33c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } health_report { overall_health: UNKNOWN } } }
Aug 08 03:51:15 dist-test-slave-jlzn krb5kdc[29731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161073, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:15.798805 30571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:15.786199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:44399 (local address 127.0.0.1:47278)
0808 03:51:15.786829 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:15.786846 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:15.786970 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:15.787253 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:15.787256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:15.787969 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:15.788237 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:15.788247 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.789282 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.789285 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:15.790266 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:15.790277 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.790997 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.791002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:15.791120 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:15.791757 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:15.791773 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:15.793915 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:15.796654 (+ 2739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:15.796664 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:15.796670 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:15.797072 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:15.797401 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:15.797404 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:15.797406 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:15.797528 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:15.798006 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:15.798012 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:15.798153 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 03:51:15.798292 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 03:51:15.798489 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":352,"threads_started":1}
I20260808 03:51:15.799003 29867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:15.786140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47278 (local address 127.26.146.190:44399)
0808 03:51:15.786337 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:15.786344 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:15.786918 (+ 574us) server_negotiation.cc:407] Connection header received
0808 03:51:15.787069 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:15.787073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:15.787134 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:15.787217 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:15.788380 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.789170 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.790402 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:15.790861 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:15.790955 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:15.794109 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:15.794114 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:15.794117 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:15.794147 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:15.796542 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:15.797183 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:15.797188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:15.797190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:15.797307 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:15.797626 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:15.797631 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:15.797633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:15.797903 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:15.798013 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:15.798376 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 03:51:15.798664 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":64}
W20260808 03:51:15.799731 30108 tablet.cc:2369] T a540a95833884776b10317172bd0b1f9 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:51:15.803673 29794 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:47278:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 03:51:15.804776 29794 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 03:51:15.806766 30324 tablet.cc:2369] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:51:15.811213 29995 tablet_service.cc:1467] Processing CreateTablet for tablet 2eea7a42e72e443fa1d9b5d191b067a9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a9b9455016d44e2ebc97c3052910cc7f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 03:51:15.811529 30216 tablet_service.cc:1467] Processing CreateTablet for tablet 2eea7a42e72e443fa1d9b5d191b067a9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a9b9455016d44e2ebc97c3052910cc7f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 03:51:15.811532 30425 tablet_service.cc:1467] Processing CreateTablet for tablet 2eea7a42e72e443fa1d9b5d191b067a9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a9b9455016d44e2ebc97c3052910cc7f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 03:51:15.812110 29995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2eea7a42e72e443fa1d9b5d191b067a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:15.812426 30425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2eea7a42e72e443fa1d9b5d191b067a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:15.812507 30216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2eea7a42e72e443fa1d9b5d191b067a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:15.817346 30551 tablet_bootstrap.cc:492] T 2eea7a42e72e443fa1d9b5d191b067a9 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Bootstrap starting.
I20260808 03:51:15.818846 30551 tablet_bootstrap.cc:654] T 2eea7a42e72e443fa1d9b5d191b067a9 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:15.819667 30550 tablet_bootstrap.cc:492] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55: Bootstrap starting.
I20260808 03:51:15.820726 30549 tablet_bootstrap.cc:492] T 2eea7a42e72e443fa1d9b5d191b067a9 P 5c3f01dda9234f2aa081ad4435bf94a1: Bootstrap starting.
I20260808 03:51:15.821352 30550 tablet_bootstrap.cc:654] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:15.821414 30551 tablet_bootstrap.cc:492] T 2eea7a42e72e443fa1d9b5d191b067a9 P 6ae07d49ebd941b2bf8e3fde33a76b6f: No bootstrap required, opened a new log
I20260808 03:51:15.821550 30551 ts_tablet_manager.cc:1397] T 2eea7a42e72e443fa1d9b5d191b067a9 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
W20260808 03:51:15.819794 30521 tablet.cc:2369] T a540a95833884776b10317172bd0b1f9 P 5c3f01dda9234f2aa081ad4435bf94a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:51:15.822307 30551 raft_consensus.cc:348] T 2eea7a42e72e443fa1d9b5d191b067a9 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:15.822589 30551 raft_consensus.cc:374] T 2eea7a42e72e443fa1d9b5d191b067a9 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:15.822731 30551 raft_consensus.cc:729] T 2eea7a42e72e443fa1d9b5d191b067a9 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ae07d49ebd941b2bf8e3fde33a76b6f, State: Initialized, Role: FOLLOWER
I20260808 03:51:15.823195 30549 tablet_bootstrap.cc:654] T 2eea7a42e72e443fa1d9b5d191b067a9 P 5c3f01dda9234f2aa081ad4435bf94a1: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:15.823045 30551 consensus_queue.cc:260] T 2eea7a42e72e443fa1d9b5d191b067a9 P 6ae07d49ebd941b2bf8e3fde33a76b6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:15.824050 30551 ts_tablet_manager.cc:1428] T 2eea7a42e72e443fa1d9b5d191b067a9 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 03:51:15.823945 30560 tablet_replica.cc:439] T 2eea7a42e72e443fa1d9b5d191b067a9 P 6ae07d49ebd941b2bf8e3fde33a76b6f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } } }
I20260808 03:51:15.824379 30560 tablet_replica.cc:442] T 2eea7a42e72e443fa1d9b5d191b067a9 P 6ae07d49ebd941b2bf8e3fde33a76b6f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 03:51:15.825081 30550 tablet_bootstrap.cc:492] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55: No bootstrap required, opened a new log
I20260808 03:51:15.825224 30550 ts_tablet_manager.cc:1397] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 03:51:15.825898 30550 raft_consensus.cc:348] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:15.826175 30550 raft_consensus.cc:374] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:15.826342 30550 raft_consensus.cc:729] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cae774377c4242fb87db8fa139f33c55, State: Initialized, Role: FOLLOWER
I20260808 03:51:15.826668 30549 tablet_bootstrap.cc:492] T 2eea7a42e72e443fa1d9b5d191b067a9 P 5c3f01dda9234f2aa081ad4435bf94a1: No bootstrap required, opened a new log
I20260808 03:51:15.826654 30550 consensus_queue.cc:260] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:15.826862 30549 ts_tablet_manager.cc:1397] T 2eea7a42e72e443fa1d9b5d191b067a9 P 5c3f01dda9234f2aa081ad4435bf94a1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 03:51:15.827088 30559 tablet_replica.cc:439] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } } }
I20260808 03:51:15.827263 30550 ts_tablet_manager.cc:1428] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 03:51:15.827414 30559 tablet_replica.cc:442] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 03:51:15.827659 30549 raft_consensus.cc:348] T 2eea7a42e72e443fa1d9b5d191b067a9 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:15.827975 30549 raft_consensus.cc:374] T 2eea7a42e72e443fa1d9b5d191b067a9 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:15.828081 30549 raft_consensus.cc:729] T 2eea7a42e72e443fa1d9b5d191b067a9 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3f01dda9234f2aa081ad4435bf94a1, State: Initialized, Role: FOLLOWER
I20260808 03:51:15.828320 30549 consensus_queue.cc:260] T 2eea7a42e72e443fa1d9b5d191b067a9 P 5c3f01dda9234f2aa081ad4435bf94a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:15.828850 30558 tablet_replica.cc:439] T 2eea7a42e72e443fa1d9b5d191b067a9 P 5c3f01dda9234f2aa081ad4435bf94a1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } } }
I20260808 03:51:15.829197 30558 tablet_replica.cc:442] T 2eea7a42e72e443fa1d9b5d191b067a9 P 5c3f01dda9234f2aa081ad4435bf94a1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 03:51:15.829617 30549 ts_tablet_manager.cc:1428] T 2eea7a42e72e443fa1d9b5d191b067a9 P 5c3f01dda9234f2aa081ad4435bf94a1: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 03:51:16.006732 30559 raft_consensus.cc:482] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:51:16.006942 30559 raft_consensus.cc:504] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:16.007577 30559 leader_election.cc:288] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ae07d49ebd941b2bf8e3fde33a76b6f (127.26.146.129:37659), 5c3f01dda9234f2aa081ad4435bf94a1 (127.26.146.131:36241)
I20260808 03:51:16.008143 30030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eea7a42e72e443fa1d9b5d191b067a9" candidate_uuid: "cae774377c4242fb87db8fa139f33c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" is_pre_election: true
I20260808 03:51:16.008235 30453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eea7a42e72e443fa1d9b5d191b067a9" candidate_uuid: "cae774377c4242fb87db8fa139f33c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" is_pre_election: true
I20260808 03:51:16.008394 30030 raft_consensus.cc:2454] T 2eea7a42e72e443fa1d9b5d191b067a9 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cae774377c4242fb87db8fa139f33c55 in term 0.
I20260808 03:51:16.008544 30453 raft_consensus.cc:2454] T 2eea7a42e72e443fa1d9b5d191b067a9 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cae774377c4242fb87db8fa139f33c55 in term 0.
I20260808 03:51:16.008837 30145 leader_election.cc:302] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ae07d49ebd941b2bf8e3fde33a76b6f, cae774377c4242fb87db8fa139f33c55; no voters:
I20260808 03:51:16.009140 30559 raft_consensus.cc:2781] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:51:16.009234 30559 raft_consensus.cc:482] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:51:16.009336 30559 raft_consensus.cc:3037] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:16.011315 30559 raft_consensus.cc:504] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:16.011860 30559 leader_election.cc:288] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55 [CANDIDATE]: Term 1 election: Requested vote from peers 6ae07d49ebd941b2bf8e3fde33a76b6f (127.26.146.129:37659), 5c3f01dda9234f2aa081ad4435bf94a1 (127.26.146.131:36241)
I20260808 03:51:16.012265 30453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eea7a42e72e443fa1d9b5d191b067a9" candidate_uuid: "cae774377c4242fb87db8fa139f33c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3f01dda9234f2aa081ad4435bf94a1"
I20260808 03:51:16.012264 30030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eea7a42e72e443fa1d9b5d191b067a9" candidate_uuid: "cae774377c4242fb87db8fa139f33c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f"
I20260808 03:51:16.012487 30030 raft_consensus.cc:3037] T 2eea7a42e72e443fa1d9b5d191b067a9 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:16.012487 30453 raft_consensus.cc:3037] T 2eea7a42e72e443fa1d9b5d191b067a9 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:16.015349 30453 raft_consensus.cc:2454] T 2eea7a42e72e443fa1d9b5d191b067a9 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cae774377c4242fb87db8fa139f33c55 in term 1.
I20260808 03:51:16.015349 30030 raft_consensus.cc:2454] T 2eea7a42e72e443fa1d9b5d191b067a9 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cae774377c4242fb87db8fa139f33c55 in term 1.
I20260808 03:51:16.015743 30143 leader_election.cc:302] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3f01dda9234f2aa081ad4435bf94a1, cae774377c4242fb87db8fa139f33c55; no voters:
I20260808 03:51:16.016033 30559 raft_consensus.cc:2781] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:16.016198 30559 raft_consensus.cc:686] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55 [term 1 LEADER]: Becoming Leader. State: Replica: cae774377c4242fb87db8fa139f33c55, State: Running, Role: LEADER
I20260808 03:51:16.016422 30559 consensus_queue.cc:237] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:16.018016 30563 tablet_replica.cc:439] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55: TxnStatusTablet state changed. Reason: New leader cae774377c4242fb87db8fa139f33c55. Latest consensus state: current_term: 1 leader_uuid: "cae774377c4242fb87db8fa139f33c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } } }
I20260808 03:51:16.018668 30563 tablet_replica.cc:442] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55: This TxnStatusTablet replica's current role is: LEADER
I20260808 03:51:16.019382 30576 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 03:51:16.019724 30576 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 03:51:16.019665 29796 catalog_manager.cc:5495] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55 reported cstate change: term changed from 0 to 1, leader changed from <none> to cae774377c4242fb87db8fa139f33c55 (127.26.146.130). New cstate: current_term: 1 leader_uuid: "cae774377c4242fb87db8fa139f33c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } health_report { overall_health: UNKNOWN } } }
I20260808 03:51:16.068686 30543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:16.061439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37360 (local address 127.26.146.130:33031)
0808 03:51:16.061649 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:16.061656 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:16.061973 (+ 317us) server_negotiation.cc:407] Connection header received
0808 03:51:16.062250 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:16.062256 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:16.062374 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:16.062543 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:51:16.062976 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.064180 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.066075 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.066741 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.066866 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:16.067408 (+ 542us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:51:16.068056 (+ 648us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:51:16.068177 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:16.068308 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 03:51:16.068403 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 03:51:16.068779 30571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:16.061470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:33031 (local address 127.0.0.1:37360)
0808 03:51:16.061871 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:16.061891 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:16.062046 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:16.062575 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:16.062580 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:16.062594 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 03:51:16.062816 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:16.062829 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.064309 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.064315 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:16.065910 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:16.065926 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.066913 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.066919 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:16.067128 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:16.067200 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 03:51:16.068164 (+ 964us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 03:51:16.068187 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 03:51:16.068282 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 03:51:16.068446 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220}
I20260808 03:51:16.076839 30453 raft_consensus.cc:1261] T 2eea7a42e72e443fa1d9b5d191b067a9 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 1 FOLLOWER]: Refusing update from remote peer cae774377c4242fb87db8fa139f33c55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:51:16.076855 30030 raft_consensus.cc:1261] T 2eea7a42e72e443fa1d9b5d191b067a9 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 1 FOLLOWER]: Refusing update from remote peer cae774377c4242fb87db8fa139f33c55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:51:16.077829 30559 consensus_queue.cc:1035] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:51:16.078126 30563 consensus_queue.cc:1035] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:51:16.087322 30560 tablet_replica.cc:439] T 2eea7a42e72e443fa1d9b5d191b067a9 P 6ae07d49ebd941b2bf8e3fde33a76b6f: TxnStatusTablet state changed. Reason: New leader cae774377c4242fb87db8fa139f33c55. Latest consensus state: current_term: 1 leader_uuid: "cae774377c4242fb87db8fa139f33c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } } }
I20260808 03:51:16.087646 30560 tablet_replica.cc:442] T 2eea7a42e72e443fa1d9b5d191b067a9 P 6ae07d49ebd941b2bf8e3fde33a76b6f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 03:51:16.089885 30558 tablet_replica.cc:439] T 2eea7a42e72e443fa1d9b5d191b067a9 P 5c3f01dda9234f2aa081ad4435bf94a1: TxnStatusTablet state changed. Reason: New leader cae774377c4242fb87db8fa139f33c55. Latest consensus state: current_term: 1 leader_uuid: "cae774377c4242fb87db8fa139f33c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } } }
I20260808 03:51:16.090144 30558 tablet_replica.cc:442] T 2eea7a42e72e443fa1d9b5d191b067a9 P 5c3f01dda9234f2aa081ad4435bf94a1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 03:51:16.092398 30585 mvcc.cc:205] Tried to move back new op lower bound from 7316115767601434624 to 7316115767365566464. Current Snapshot: MvccSnapshot[applied={T|T < 7316115767601434624}]
I20260808 03:51:16.093132 30559 tablet_replica.cc:439] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cae774377c4242fb87db8fa139f33c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } } }
I20260808 03:51:16.094036 30559 tablet_replica.cc:442] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55: This TxnStatusTablet replica's current role is: LEADER
I20260808 03:51:16.094251 30559 tablet_replica.cc:439] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cae774377c4242fb87db8fa139f33c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } } }
I20260808 03:51:16.094476 30559 tablet_replica.cc:442] T 2eea7a42e72e443fa1d9b5d191b067a9 P cae774377c4242fb87db8fa139f33c55: This TxnStatusTablet replica's current role is: LEADER
I20260808 03:51:16.094775 30558 tablet_replica.cc:439] T 2eea7a42e72e443fa1d9b5d191b067a9 P 5c3f01dda9234f2aa081ad4435bf94a1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cae774377c4242fb87db8fa139f33c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } } }
I20260808 03:51:16.094985 30558 tablet_replica.cc:442] T 2eea7a42e72e443fa1d9b5d191b067a9 P 5c3f01dda9234f2aa081ad4435bf94a1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 03:51:16.107513 30560 tablet_replica.cc:439] T 2eea7a42e72e443fa1d9b5d191b067a9 P 6ae07d49ebd941b2bf8e3fde33a76b6f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cae774377c4242fb87db8fa139f33c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } } }
I20260808 03:51:16.107825 30560 tablet_replica.cc:442] T 2eea7a42e72e443fa1d9b5d191b067a9 P 6ae07d49ebd941b2bf8e3fde33a76b6f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 03:51:16.115636 30543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:16.110986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37362 (local address 127.26.146.130:33031)
0808 03:51:16.111169 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:16.111175 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:16.111192 (+ 17us) server_negotiation.cc:407] Connection header received
0808 03:51:16.111271 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:16.111276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:16.111360 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:16.111454 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:51:16.111834 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.112673 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.113981 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.114402 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.114511 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:16.114917 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:51:16.115045 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:51:16.115116 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:16.115265 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 03:51:16.115342 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 03:51:16.132346 30543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:16.126150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:47435 (local address 127.26.146.130:33031)
0808 03:51:16.126350 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:16.126356 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:16.126967 (+ 611us) server_negotiation.cc:407] Connection header received
0808 03:51:16.127133 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:16.127138 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:16.127223 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:16.127320 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:51:16.127830 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.128939 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.130346 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.131012 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.131131 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:16.131535 (+ 404us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:51:16.131684 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:51:16.131765 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:16.131945 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 03:51:16.132026 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 03:51:16.132427 30601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:16.126205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:33031 (local address 127.26.146.130:47435)
0808 03:51:16.126877 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:16.126894 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:16.127007 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:16.127325 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:16.127328 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:16.127337 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 03:51:16.127671 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:16.127681 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.129051 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.129054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:16.130215 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:16.130227 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.131170 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.131175 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:16.131300 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:16.131350 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 03:51:16.131797 (+ 447us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 03:51:16.131830 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 03:51:16.131894 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 03:51:16.132000 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":349,"threads_started":1}
I20260808 03:51:16.141191 30453 raft_consensus.cc:1261] T a540a95833884776b10317172bd0b1f9 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 1 FOLLOWER]: Refusing update from remote peer cae774377c4242fb87db8fa139f33c55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:51:16.141237 30030 raft_consensus.cc:1261] T a540a95833884776b10317172bd0b1f9 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 1 FOLLOWER]: Refusing update from remote peer cae774377c4242fb87db8fa139f33c55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:51:16.141750 30559 consensus_queue.cc:1035] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:51:16.141999 30563 consensus_queue.cc:1035] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:51:16.155135 30587 mvcc.cc:205] Tried to move back new op lower bound from 7316115767869685760 to 7316115766269132800. Current Snapshot: MvccSnapshot[applied={T|T < 7316115767869685760}]
I20260808 03:51:16.201635 29796 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:51:16.201887 29796 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:51:16.205153 29796 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e: Sending DeleteTablet for 3 replicas of tablet a540a95833884776b10317172bd0b1f9
I20260808 03:51:16.206269 30215 tablet_service.cc:1514] Processing DeleteTablet for tablet a540a95833884776b10317172bd0b1f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:16 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:37352
I20260808 03:51:16.206341 30425 tablet_service.cc:1514] Processing DeleteTablet for tablet a540a95833884776b10317172bd0b1f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:16 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:56228
I20260808 03:51:16.206987 29995 tablet_service.cc:1514] Processing DeleteTablet for tablet a540a95833884776b10317172bd0b1f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:16 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:46470
I20260808 03:51:16.207161 30607 tablet_replica.cc:330] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55: stopping tablet replica
I20260808 03:51:16.207640 30607 raft_consensus.cc:2229] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55 [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:51:16.208482 30607 raft_consensus.cc:2258] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:16.212065 30608 tablet_replica.cc:330] T a540a95833884776b10317172bd0b1f9 P 5c3f01dda9234f2aa081ad4435bf94a1: stopping tablet replica
I20260808 03:51:16.212425 30608 raft_consensus.cc:2229] T a540a95833884776b10317172bd0b1f9 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:51:16.212798 30608 raft_consensus.cc:2258] T a540a95833884776b10317172bd0b1f9 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:16.214531 30607 ts_tablet_manager.cc:1905] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:16.214690 30608 ts_tablet_manager.cc:1905] T a540a95833884776b10317172bd0b1f9 P 5c3f01dda9234f2aa081ad4435bf94a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:16.216369 29796 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47264:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:51:16.217165 29796 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:51:16.224478 30609 tablet_replica.cc:330] T a540a95833884776b10317172bd0b1f9 P 6ae07d49ebd941b2bf8e3fde33a76b6f: stopping tablet replica
I20260808 03:51:16.224754 30425 tablet_service.cc:1467] Processing CreateTablet for tablet a192dc8fcf3848e38e2638bbed44cc98 (DEFAULT_TABLE table=test-table [id=1b584316a7ba4f51a86d76aaf8bda55d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:16.224843 30215 tablet_service.cc:1467] Processing CreateTablet for tablet a192dc8fcf3848e38e2638bbed44cc98 (DEFAULT_TABLE table=test-table [id=1b584316a7ba4f51a86d76aaf8bda55d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:16.225183 30425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a192dc8fcf3848e38e2638bbed44cc98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:16.225317 30215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a192dc8fcf3848e38e2638bbed44cc98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:16.225433 30609 raft_consensus.cc:2229] T a540a95833884776b10317172bd0b1f9 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:51:16.225804 30609 raft_consensus.cc:2258] T a540a95833884776b10317172bd0b1f9 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:16.224917 29995 tablet_service.cc:1467] Processing CreateTablet for tablet a192dc8fcf3848e38e2638bbed44cc98 (DEFAULT_TABLE table=test-table [id=1b584316a7ba4f51a86d76aaf8bda55d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:16.226426 29995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a192dc8fcf3848e38e2638bbed44cc98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:16.228941 30609 ts_tablet_manager.cc:1905] T a540a95833884776b10317172bd0b1f9 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:16.231243 30549 tablet_bootstrap.cc:492] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1: Bootstrap starting.
I20260808 03:51:16.232034 30550 tablet_bootstrap.cc:492] T a192dc8fcf3848e38e2638bbed44cc98 P cae774377c4242fb87db8fa139f33c55: Bootstrap starting.
I20260808 03:51:16.232854 30549 tablet_bootstrap.cc:654] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:16.233767 30550 tablet_bootstrap.cc:654] T a192dc8fcf3848e38e2638bbed44cc98 P cae774377c4242fb87db8fa139f33c55: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:16.240823 30607 ts_tablet_manager.cc:1918] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 03:51:16.241066 30607 log.cc:1196] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/a540a95833884776b10317172bd0b1f9
I20260808 03:51:16.241870 30551 tablet_bootstrap.cc:492] T a192dc8fcf3848e38e2638bbed44cc98 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Bootstrap starting.
I20260808 03:51:16.241500 30550 tablet_bootstrap.cc:492] T a192dc8fcf3848e38e2638bbed44cc98 P cae774377c4242fb87db8fa139f33c55: No bootstrap required, opened a new log
I20260808 03:51:16.242167 30550 ts_tablet_manager.cc:1397] T a192dc8fcf3848e38e2638bbed44cc98 P cae774377c4242fb87db8fa139f33c55: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.004s
I20260808 03:51:16.242514 30607 ts_tablet_manager.cc:1939] T a540a95833884776b10317172bd0b1f9 P cae774377c4242fb87db8fa139f33c55: Deleting consensus metadata
I20260808 03:51:16.242938 30550 raft_consensus.cc:348] T a192dc8fcf3848e38e2638bbed44cc98 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:16.243184 30550 raft_consensus.cc:374] T a192dc8fcf3848e38e2638bbed44cc98 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:16.243271 30550 raft_consensus.cc:729] T a192dc8fcf3848e38e2638bbed44cc98 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cae774377c4242fb87db8fa139f33c55, State: Initialized, Role: FOLLOWER
I20260808 03:51:16.243420 30551 tablet_bootstrap.cc:654] T a192dc8fcf3848e38e2638bbed44cc98 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:16.243536 30550 consensus_queue.cc:260] T a192dc8fcf3848e38e2638bbed44cc98 P cae774377c4242fb87db8fa139f33c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:16.244187 30550 ts_tablet_manager.cc:1428] T a192dc8fcf3848e38e2638bbed44cc98 P cae774377c4242fb87db8fa139f33c55: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260808 03:51:16.244347 29775 catalog_manager.cc:4841] TS cae774377c4242fb87db8fa139f33c55 (127.26.146.130:33031): tablet a540a95833884776b10317172bd0b1f9 (table test-table [id=ea02461482b84ad8b575f6628081895a]) successfully deleted
I20260808 03:51:16.247658 30609 ts_tablet_manager.cc:1918] T a540a95833884776b10317172bd0b1f9 P 6ae07d49ebd941b2bf8e3fde33a76b6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 03:51:16.248104 30609 log.cc:1196] T a540a95833884776b10317172bd0b1f9 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-0/wal/wals/a540a95833884776b10317172bd0b1f9
I20260808 03:51:16.248685 30609 ts_tablet_manager.cc:1939] T a540a95833884776b10317172bd0b1f9 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Deleting consensus metadata
I20260808 03:51:16.250476 29779 catalog_manager.cc:4841] TS 6ae07d49ebd941b2bf8e3fde33a76b6f (127.26.146.129:37659): tablet a540a95833884776b10317172bd0b1f9 (table test-table [id=ea02461482b84ad8b575f6628081895a]) successfully deleted
I20260808 03:51:16.251608 30608 ts_tablet_manager.cc:1918] T a540a95833884776b10317172bd0b1f9 P 5c3f01dda9234f2aa081ad4435bf94a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 03:51:16.251868 30608 log.cc:1196] T a540a95833884776b10317172bd0b1f9 P 5c3f01dda9234f2aa081ad4435bf94a1: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/a540a95833884776b10317172bd0b1f9
I20260808 03:51:16.252080 30549 tablet_bootstrap.cc:492] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1: No bootstrap required, opened a new log
I20260808 03:51:16.252079 30551 tablet_bootstrap.cc:492] T a192dc8fcf3848e38e2638bbed44cc98 P 6ae07d49ebd941b2bf8e3fde33a76b6f: No bootstrap required, opened a new log
I20260808 03:51:16.252246 30551 ts_tablet_manager.cc:1397] T a192dc8fcf3848e38e2638bbed44cc98 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260808 03:51:16.252313 30549 ts_tablet_manager.cc:1397] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.005s
I20260808 03:51:16.252455 30608 ts_tablet_manager.cc:1939] T a540a95833884776b10317172bd0b1f9 P 5c3f01dda9234f2aa081ad4435bf94a1: Deleting consensus metadata
I20260808 03:51:16.252990 30551 raft_consensus.cc:348] T a192dc8fcf3848e38e2638bbed44cc98 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:16.253098 30549 raft_consensus.cc:348] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:16.253273 30551 raft_consensus.cc:374] T a192dc8fcf3848e38e2638bbed44cc98 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:16.253329 30549 raft_consensus.cc:374] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:16.253373 30551 raft_consensus.cc:729] T a192dc8fcf3848e38e2638bbed44cc98 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ae07d49ebd941b2bf8e3fde33a76b6f, State: Initialized, Role: FOLLOWER
I20260808 03:51:16.253479 30549 raft_consensus.cc:729] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3f01dda9234f2aa081ad4435bf94a1, State: Initialized, Role: FOLLOWER
I20260808 03:51:16.253649 30551 consensus_queue.cc:260] T a192dc8fcf3848e38e2638bbed44cc98 P 6ae07d49ebd941b2bf8e3fde33a76b6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:16.253850 30549 consensus_queue.cc:260] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:16.254930 30551 ts_tablet_manager.cc:1428] T a192dc8fcf3848e38e2638bbed44cc98 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260808 03:51:16.255736 30549 ts_tablet_manager.cc:1428] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260808 03:51:16.256414 29777 catalog_manager.cc:4841] TS 5c3f01dda9234f2aa081ad4435bf94a1 (127.26.146.131:36241): tablet a540a95833884776b10317172bd0b1f9 (table test-table [id=ea02461482b84ad8b575f6628081895a]) successfully deleted
I20260808 03:51:16.275174 30558 raft_consensus.cc:482] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:51:16.275413 30558 raft_consensus.cc:504] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:16.277243 30558 leader_election.cc:288] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ae07d49ebd941b2bf8e3fde33a76b6f (127.26.146.129:37659), cae774377c4242fb87db8fa139f33c55 (127.26.146.130:33031)
I20260808 03:51:16.285300 30619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:16.278027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:58437 (local address 127.26.146.129:37659)
0808 03:51:16.278972 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:16.278979 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:16.278998 (+ 19us) server_negotiation.cc:407] Connection header received
0808 03:51:16.279069 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:16.279074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:16.279148 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:16.279266 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:16.279795 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.281071 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.282987 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.284052 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.284292 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:16.284371 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:16.284791 (+ 420us) server_negotiation.cc:299] Negotiation successful
0808 03:51:16.284925 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":810,"thread_start_us":656,"threads_started":1}
I20260808 03:51:16.285356 30615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:16.277490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:37659 (local address 127.26.146.131:58437)
0808 03:51:16.278314 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:16.278339 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:16.278472 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:16.279341 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:16.279346 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:16.279368 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:16.279635 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:16.279648 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.281273 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.281279 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:16.282833 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:16.282845 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.284466 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.284472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:16.284633 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:16.284669 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 03:51:16.284764 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 03:51:16.284896 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"mutex_wait_us":50,"thread_start_us":654,"threads_started":1}
I20260808 03:51:16.286103 30030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192dc8fcf3848e38e2638bbed44cc98" candidate_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" is_pre_election: true
I20260808 03:51:16.286386 30030 raft_consensus.cc:2454] T a192dc8fcf3848e38e2638bbed44cc98 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3f01dda9234f2aa081ad4435bf94a1 in term 0.
I20260808 03:51:16.286583 30618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:16.277602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:33031 (local address 127.26.146.131:46257)
0808 03:51:16.279157 (+ 1555us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:16.279172 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:16.279310 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:16.279918 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:16.279921 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:16.279942 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:16.280221 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:16.280230 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.281466 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.281472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:16.283582 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:16.283598 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.285675 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.285681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:16.285884 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:16.285918 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 03:51:16.286060 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 03:51:16.286186 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1440,"mutex_wait_us":1,"thread_start_us":1306,"threads_started":1}
I20260808 03:51:16.287072 30355 leader_election.cc:302] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3f01dda9234f2aa081ad4435bf94a1, 6ae07d49ebd941b2bf8e3fde33a76b6f; no voters:
I20260808 03:51:16.287288 30543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:16.277687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:46257 (local address 127.26.146.130:33031)
0808 03:51:16.277875 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:16.277882 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:16.279265 (+ 1383us) server_negotiation.cc:407] Connection header received
0808 03:51:16.279491 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:16.279497 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:16.279602 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:16.279694 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:16.280380 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.281306 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.283783 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.285258 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.285409 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:16.285485 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:16.286110 (+ 625us) server_negotiation.cc:299] Negotiation successful
0808 03:51:16.287012 (+ 902us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 03:51:16.287441 30558 raft_consensus.cc:2781] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:51:16.287535 30558 raft_consensus.cc:482] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:51:16.287599 30558 raft_consensus.cc:3037] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:16.288084 30258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192dc8fcf3848e38e2638bbed44cc98" candidate_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cae774377c4242fb87db8fa139f33c55" is_pre_election: true
I20260808 03:51:16.288537 30258 raft_consensus.cc:2454] T a192dc8fcf3848e38e2638bbed44cc98 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3f01dda9234f2aa081ad4435bf94a1 in term 0.
I20260808 03:51:16.290230 30558 raft_consensus.cc:504] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:16.290827 30558 leader_election.cc:288] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1 [CANDIDATE]: Term 1 election: Requested vote from peers 6ae07d49ebd941b2bf8e3fde33a76b6f (127.26.146.129:37659), cae774377c4242fb87db8fa139f33c55 (127.26.146.130:33031)
I20260808 03:51:16.291201 30030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192dc8fcf3848e38e2638bbed44cc98" candidate_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f"
I20260808 03:51:16.291406 30030 raft_consensus.cc:3037] T a192dc8fcf3848e38e2638bbed44cc98 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:16.291396 30258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192dc8fcf3848e38e2638bbed44cc98" candidate_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cae774377c4242fb87db8fa139f33c55"
I20260808 03:51:16.291607 30258 raft_consensus.cc:3037] T a192dc8fcf3848e38e2638bbed44cc98 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:16.294304 30030 raft_consensus.cc:2454] T a192dc8fcf3848e38e2638bbed44cc98 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3f01dda9234f2aa081ad4435bf94a1 in term 1.
I20260808 03:51:16.294734 30258 raft_consensus.cc:2454] T a192dc8fcf3848e38e2638bbed44cc98 P cae774377c4242fb87db8fa139f33c55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3f01dda9234f2aa081ad4435bf94a1 in term 1.
I20260808 03:51:16.294726 30355 leader_election.cc:302] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3f01dda9234f2aa081ad4435bf94a1, 6ae07d49ebd941b2bf8e3fde33a76b6f; no voters:
I20260808 03:51:16.294994 30558 raft_consensus.cc:2781] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:16.295746 30558 raft_consensus.cc:686] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3f01dda9234f2aa081ad4435bf94a1, State: Running, Role: LEADER
I20260808 03:51:16.296126 30558 consensus_queue.cc:237] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:16.299690 29797 catalog_manager.cc:5495] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3f01dda9234f2aa081ad4435bf94a1 (127.26.146.131). New cstate: current_term: 1 leader_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } health_report { overall_health: HEALTHY } } }
I20260808 03:51:16.330513 30625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:16.320796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56236 (local address 127.26.146.131:36241)
0808 03:51:16.324521 (+ 3725us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:16.324526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:16.324550 (+ 24us) server_negotiation.cc:407] Connection header received
0808 03:51:16.324640 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:16.324644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:16.324753 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:16.324924 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:51:16.325292 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.326480 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.327828 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.328515 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.328644 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:16.329064 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:51:16.329827 (+ 763us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:51:16.329980 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:16.330041 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 03:51:16.330125 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3597,"thread_start_us":3403,"threads_started":1}
I20260808 03:51:16.346863 30630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:16.340955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:33031 (local address 127.26.146.131:57379)
0808 03:51:16.341568 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:16.341583 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:16.341697 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:16.342030 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:16.342032 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:16.342041 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 03:51:16.342305 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:16.342312 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.343721 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.343725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:16.344792 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:16.344804 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.345765 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.345769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:16.345907 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:16.345957 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 03:51:16.346416 (+ 459us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 03:51:16.346425 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 03:51:16.346486 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 03:51:16.346582 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":312,"threads_started":1}
I20260808 03:51:16.346915 30543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:16.340890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:57379 (local address 127.26.146.130:33031)
0808 03:51:16.341059 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:16.341064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:16.341673 (+ 609us) server_negotiation.cc:407] Connection header received
0808 03:51:16.341819 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:16.341825 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:16.341924 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:16.342033 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:51:16.342446 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.343595 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.344934 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.345600 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.345749 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:16.346149 (+ 400us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:51:16.346322 (+ 173us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:51:16.346414 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:16.346532 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 03:51:16.346613 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 03:51:16.355705 30030 raft_consensus.cc:1261] T a192dc8fcf3848e38e2638bbed44cc98 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 1 FOLLOWER]: Refusing update from remote peer 5c3f01dda9234f2aa081ad4435bf94a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:51:16.356097 30258 raft_consensus.cc:1261] T a192dc8fcf3848e38e2638bbed44cc98 P cae774377c4242fb87db8fa139f33c55 [term 1 FOLLOWER]: Refusing update from remote peer 5c3f01dda9234f2aa081ad4435bf94a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:51:16.356408 30620 consensus_queue.cc:1035] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:51:16.356794 30558 consensus_queue.cc:1035] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:51:16.363848 30586 mvcc.cc:205] Tried to move back new op lower bound from 7316115768747454464 to 7316115768512393216. Current Snapshot: MvccSnapshot[applied={T|T < 7316115768747454464}]
I20260808 03:51:16.396003 30625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:16.390672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:52759 (local address 127.26.146.131:36241)
0808 03:51:16.390835 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:16.390840 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:16.391248 (+ 408us) server_negotiation.cc:407] Connection header received
0808 03:51:16.391320 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:16.391325 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:16.391405 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:16.391514 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:51:16.391969 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.392816 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.394194 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.394718 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.394835 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:16.395231 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:51:16.395363 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:51:16.395487 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:16.395621 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 03:51:16.395695 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 03:51:16.396049 30601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:16.390774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:36241 (local address 127.26.146.130:52759)
0808 03:51:16.390969 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:16.390989 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:16.391115 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:16.391555 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:16.391560 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:16.391573 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 03:51:16.391828 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:16.391842 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.392943 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.392949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:16.394050 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:16.394062 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:16.394849 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:16.394859 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:16.395014 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:16.395053 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 03:51:16.395502 (+ 449us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 03:51:16.395514 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 03:51:16.395588 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 03:51:16.395734 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 03:51:16.397410 30586 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 03:51:18.010912 29797 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:51:18.011152 29797 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:51:18.015264 29797 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e: Sending DeleteTablet for 3 replicas of tablet a192dc8fcf3848e38e2638bbed44cc98
I20260808 03:51:18.016152 30425 tablet_service.cc:1514] Processing DeleteTablet for tablet a192dc8fcf3848e38e2638bbed44cc98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:18 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:56228
I20260808 03:51:18.016166 30215 tablet_service.cc:1514] Processing DeleteTablet for tablet a192dc8fcf3848e38e2638bbed44cc98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:18 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:37352
I20260808 03:51:18.016168 29995 tablet_service.cc:1514] Processing DeleteTablet for tablet a192dc8fcf3848e38e2638bbed44cc98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:18 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:46470
I20260808 03:51:18.026331 30668 tablet_replica.cc:330] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1: stopping tablet replica
I20260808 03:51:18.026665 30668 raft_consensus.cc:2229] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:51:18.027300 30668 raft_consensus.cc:2258] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:18.028213 30670 tablet_replica.cc:330] T a192dc8fcf3848e38e2638bbed44cc98 P cae774377c4242fb87db8fa139f33c55: stopping tablet replica
I20260808 03:51:18.028306 30669 tablet_replica.cc:330] T a192dc8fcf3848e38e2638bbed44cc98 P 6ae07d49ebd941b2bf8e3fde33a76b6f: stopping tablet replica
I20260808 03:51:18.028529 30668 ts_tablet_manager.cc:1905] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:18.028575 30669 raft_consensus.cc:2229] T a192dc8fcf3848e38e2638bbed44cc98 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:51:18.028553 30670 raft_consensus.cc:2229] T a192dc8fcf3848e38e2638bbed44cc98 P cae774377c4242fb87db8fa139f33c55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:51:18.028875 30669 raft_consensus.cc:2258] T a192dc8fcf3848e38e2638bbed44cc98 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:18.028883 30670 raft_consensus.cc:2258] T a192dc8fcf3848e38e2638bbed44cc98 P cae774377c4242fb87db8fa139f33c55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:18.029875 30670 ts_tablet_manager.cc:1905] T a192dc8fcf3848e38e2638bbed44cc98 P cae774377c4242fb87db8fa139f33c55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:18.030871 30669 ts_tablet_manager.cc:1905] T a192dc8fcf3848e38e2638bbed44cc98 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:18.035279 30668 ts_tablet_manager.cc:1918] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 03:51:18.035501 30668 log.cc:1196] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/a192dc8fcf3848e38e2638bbed44cc98
I20260808 03:51:18.036031 30668 ts_tablet_manager.cc:1939] T a192dc8fcf3848e38e2638bbed44cc98 P 5c3f01dda9234f2aa081ad4435bf94a1: Deleting consensus metadata
I20260808 03:51:18.036449 30670 ts_tablet_manager.cc:1918] T a192dc8fcf3848e38e2638bbed44cc98 P cae774377c4242fb87db8fa139f33c55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 03:51:18.036581 30670 log.cc:1196] T a192dc8fcf3848e38e2638bbed44cc98 P cae774377c4242fb87db8fa139f33c55: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/a192dc8fcf3848e38e2638bbed44cc98
I20260808 03:51:18.037112 30670 ts_tablet_manager.cc:1939] T a192dc8fcf3848e38e2638bbed44cc98 P cae774377c4242fb87db8fa139f33c55: Deleting consensus metadata
I20260808 03:51:18.037140 30669 ts_tablet_manager.cc:1918] T a192dc8fcf3848e38e2638bbed44cc98 P 6ae07d49ebd941b2bf8e3fde33a76b6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 03:51:18.037252 30669 log.cc:1196] T a192dc8fcf3848e38e2638bbed44cc98 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-0/wal/wals/a192dc8fcf3848e38e2638bbed44cc98
I20260808 03:51:18.037462 29777 catalog_manager.cc:4841] TS 5c3f01dda9234f2aa081ad4435bf94a1 (127.26.146.131:36241): tablet a192dc8fcf3848e38e2638bbed44cc98 (table test-table [id=1b584316a7ba4f51a86d76aaf8bda55d]) successfully deleted
I20260808 03:51:18.037621 30669 ts_tablet_manager.cc:1939] T a192dc8fcf3848e38e2638bbed44cc98 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Deleting consensus metadata
I20260808 03:51:18.038122 29775 catalog_manager.cc:4841] TS cae774377c4242fb87db8fa139f33c55 (127.26.146.130:33031): tablet a192dc8fcf3848e38e2638bbed44cc98 (table test-table [id=1b584316a7ba4f51a86d76aaf8bda55d]) successfully deleted
I20260808 03:51:18.038630 29779 catalog_manager.cc:4841] TS 6ae07d49ebd941b2bf8e3fde33a76b6f (127.26.146.129:37659): tablet a192dc8fcf3848e38e2638bbed44cc98 (table test-table [id=1b584316a7ba4f51a86d76aaf8bda55d]) successfully deleted
Aug 08 03:51:18 dist-test-slave-jlzn krb5kdc[29731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161078, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 03:51:18 dist-test-slave-jlzn krb5kdc[29731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161078, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:18.075968 30682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:18.062254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47284 (local address 127.26.146.190:44399)
0808 03:51:18.063027 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:18.063031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:18.063051 (+ 20us) server_negotiation.cc:407] Connection header received
0808 03:51:18.063103 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:18.063107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:18.063201 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:18.063298 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:18.064667 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:18.065441 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:18.066745 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:18.067128 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:18.067242 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:18.071048 (+ 3806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:18.071053 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:18.071056 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:18.071089 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:18.073461 (+ 2372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:18.074072 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:18.074077 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:18.074079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:18.074208 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:18.074541 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:18.074545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:18.074547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:18.074820 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:18.075040 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:18.075357 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 03:51:18.075597 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":493,"threads_started":1}
I20260808 03:51:18.079113 29797 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47284:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:51:18.079777 29797 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:51:18.085912 29995 tablet_service.cc:1467] Processing CreateTablet for tablet fb14261ada0949f6a7cb5a14996b8ff3 (DEFAULT_TABLE table=test-table [id=ea870b5ba0ef4fa282effe881a13e12d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:18.086149 30425 tablet_service.cc:1467] Processing CreateTablet for tablet fb14261ada0949f6a7cb5a14996b8ff3 (DEFAULT_TABLE table=test-table [id=ea870b5ba0ef4fa282effe881a13e12d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:18.086110 30215 tablet_service.cc:1467] Processing CreateTablet for tablet fb14261ada0949f6a7cb5a14996b8ff3 (DEFAULT_TABLE table=test-table [id=ea870b5ba0ef4fa282effe881a13e12d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:18.086325 29995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb14261ada0949f6a7cb5a14996b8ff3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:18.086545 30425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb14261ada0949f6a7cb5a14996b8ff3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:18.086557 30215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb14261ada0949f6a7cb5a14996b8ff3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:18.091286 30683 tablet_bootstrap.cc:492] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Bootstrap starting.
I20260808 03:51:18.092710 30683 tablet_bootstrap.cc:654] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:18.093055 30687 tablet_bootstrap.cc:492] T fb14261ada0949f6a7cb5a14996b8ff3 P cae774377c4242fb87db8fa139f33c55: Bootstrap starting.
I20260808 03:51:18.094568 30685 tablet_bootstrap.cc:492] T fb14261ada0949f6a7cb5a14996b8ff3 P 5c3f01dda9234f2aa081ad4435bf94a1: Bootstrap starting.
I20260808 03:51:18.094738 30687 tablet_bootstrap.cc:654] T fb14261ada0949f6a7cb5a14996b8ff3 P cae774377c4242fb87db8fa139f33c55: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:18.096144 30685 tablet_bootstrap.cc:654] T fb14261ada0949f6a7cb5a14996b8ff3 P 5c3f01dda9234f2aa081ad4435bf94a1: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:18.096921 30687 tablet_bootstrap.cc:492] T fb14261ada0949f6a7cb5a14996b8ff3 P cae774377c4242fb87db8fa139f33c55: No bootstrap required, opened a new log
I20260808 03:51:18.097078 30687 ts_tablet_manager.cc:1397] T fb14261ada0949f6a7cb5a14996b8ff3 P cae774377c4242fb87db8fa139f33c55: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 03:51:18.097715 30683 tablet_bootstrap.cc:492] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f: No bootstrap required, opened a new log
I20260808 03:51:18.097863 30683 ts_tablet_manager.cc:1397] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260808 03:51:18.097879 30687 raft_consensus.cc:348] T fb14261ada0949f6a7cb5a14996b8ff3 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:18.098119 30687 raft_consensus.cc:374] T fb14261ada0949f6a7cb5a14996b8ff3 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:18.098218 30687 raft_consensus.cc:729] T fb14261ada0949f6a7cb5a14996b8ff3 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cae774377c4242fb87db8fa139f33c55, State: Initialized, Role: FOLLOWER
I20260808 03:51:18.098454 30687 consensus_queue.cc:260] T fb14261ada0949f6a7cb5a14996b8ff3 P cae774377c4242fb87db8fa139f33c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:18.098630 30683 raft_consensus.cc:348] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:18.098872 30683 raft_consensus.cc:374] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:18.098961 30683 raft_consensus.cc:729] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ae07d49ebd941b2bf8e3fde33a76b6f, State: Initialized, Role: FOLLOWER
I20260808 03:51:18.099148 30687 ts_tablet_manager.cc:1428] T fb14261ada0949f6a7cb5a14996b8ff3 P cae774377c4242fb87db8fa139f33c55: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 03:51:18.099192 30683 consensus_queue.cc:260] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:18.099960 30683 ts_tablet_manager.cc:1428] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 03:51:18.100669 30685 tablet_bootstrap.cc:492] T fb14261ada0949f6a7cb5a14996b8ff3 P 5c3f01dda9234f2aa081ad4435bf94a1: No bootstrap required, opened a new log
I20260808 03:51:18.100822 30685 ts_tablet_manager.cc:1397] T fb14261ada0949f6a7cb5a14996b8ff3 P 5c3f01dda9234f2aa081ad4435bf94a1: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260808 03:51:18.101490 30685 raft_consensus.cc:348] T fb14261ada0949f6a7cb5a14996b8ff3 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:18.101713 30685 raft_consensus.cc:374] T fb14261ada0949f6a7cb5a14996b8ff3 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:18.101799 30685 raft_consensus.cc:729] T fb14261ada0949f6a7cb5a14996b8ff3 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3f01dda9234f2aa081ad4435bf94a1, State: Initialized, Role: FOLLOWER
I20260808 03:51:18.101990 30685 consensus_queue.cc:260] T fb14261ada0949f6a7cb5a14996b8ff3 P 5c3f01dda9234f2aa081ad4435bf94a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:18.103397 30685 ts_tablet_manager.cc:1428] T fb14261ada0949f6a7cb5a14996b8ff3 P 5c3f01dda9234f2aa081ad4435bf94a1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 03:51:18.103588 30691 raft_consensus.cc:482] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:51:18.103793 30691 raft_consensus.cc:504] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:18.105707 30691 leader_election.cc:288] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cae774377c4242fb87db8fa139f33c55 (127.26.146.130:33031), 5c3f01dda9234f2aa081ad4435bf94a1 (127.26.146.131:36241)
I20260808 03:51:18.122414 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:18.106528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:46017 (local address 127.26.146.131:36241)
0808 03:51:18.107471 (+ 943us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:18.107477 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:18.117019 (+ 9542us) server_negotiation.cc:407] Connection header received
0808 03:51:18.117145 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:18.117150 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:18.117222 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:18.117330 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:18.118217 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:18.118977 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:18.120740 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:18.121522 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:18.121668 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:18.121744 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:18.122058 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 03:51:18.122169 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":812,"thread_start_us":698,"threads_started":1}
I20260808 03:51:18.122454 30694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:18.105917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:33031 (local address 127.26.146.129:43677)
0808 03:51:18.116582 (+ 10665us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:18.116607 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:18.116868 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:18.117246 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:18.117250 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:18.117288 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:18.117633 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:18.117648 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:18.118858 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:18.118863 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:18.120460 (+ 1597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:18.120473 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:18.121588 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:18.121594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:18.121751 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:18.121775 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 03:51:18.121857 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 03:51:18.122059 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10511,"thread_start_us":10596,"threads_started":1}
I20260808 03:51:18.122572 30695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:18.106265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:43677 (local address 127.26.146.130:33031)
0808 03:51:18.116334 (+ 10069us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:18.116341 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:18.116788 (+ 447us) server_negotiation.cc:407] Connection header received
0808 03:51:18.117003 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:18.117008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:18.117087 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:18.117198 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:18.117814 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:18.118721 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:18.120620 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:18.121412 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:18.121571 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:18.121658 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:18.121918 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 03:51:18.122164 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9908,"thread_start_us":9762,"threads_started":1}
I20260808 03:51:18.123163 30693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:18.106065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:36241 (local address 127.26.146.129:46017)
0808 03:51:18.116920 (+ 10855us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:18.116936 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:18.117045 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:18.117541 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:18.117545 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:18.117573 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:18.117982 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:18.117995 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:18.119103 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:18.119113 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:18.120583 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:18.120596 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:18.121732 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:18.121738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:18.121879 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:18.121905 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 03:51:18.122658 (+ 753us) client_negotiation.cc:240] Negotiation successful
0808 03:51:18.122786 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10717,"thread_start_us":10217,"threads_started":1}
I20260808 03:51:18.123481 30258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb14261ada0949f6a7cb5a14996b8ff3" candidate_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cae774377c4242fb87db8fa139f33c55" is_pre_election: true
I20260808 03:51:18.123778 30258 raft_consensus.cc:2454] T fb14261ada0949f6a7cb5a14996b8ff3 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ae07d49ebd941b2bf8e3fde33a76b6f in term 0.
I20260808 03:51:18.123903 30452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb14261ada0949f6a7cb5a14996b8ff3" candidate_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" is_pre_election: true
I20260808 03:51:18.124148 30452 raft_consensus.cc:2454] T fb14261ada0949f6a7cb5a14996b8ff3 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ae07d49ebd941b2bf8e3fde33a76b6f in term 0.
I20260808 03:51:18.124395 29927 leader_election.cc:302] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ae07d49ebd941b2bf8e3fde33a76b6f, cae774377c4242fb87db8fa139f33c55; no voters:
I20260808 03:51:18.124779 30691 raft_consensus.cc:2781] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:51:18.124907 30691 raft_consensus.cc:482] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:51:18.125006 30691 raft_consensus.cc:3037] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:18.127271 30691 raft_consensus.cc:504] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:18.127858 30691 leader_election.cc:288] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f [CANDIDATE]: Term 1 election: Requested vote from peers cae774377c4242fb87db8fa139f33c55 (127.26.146.130:33031), 5c3f01dda9234f2aa081ad4435bf94a1 (127.26.146.131:36241)
I20260808 03:51:18.128412 30452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb14261ada0949f6a7cb5a14996b8ff3" candidate_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3f01dda9234f2aa081ad4435bf94a1"
I20260808 03:51:18.128630 30452 raft_consensus.cc:3037] T fb14261ada0949f6a7cb5a14996b8ff3 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:18.128713 30258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb14261ada0949f6a7cb5a14996b8ff3" candidate_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cae774377c4242fb87db8fa139f33c55"
I20260808 03:51:18.128893 30258 raft_consensus.cc:3037] T fb14261ada0949f6a7cb5a14996b8ff3 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:18.130712 30452 raft_consensus.cc:2454] T fb14261ada0949f6a7cb5a14996b8ff3 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ae07d49ebd941b2bf8e3fde33a76b6f in term 1.
I20260808 03:51:18.130769 30258 raft_consensus.cc:2454] T fb14261ada0949f6a7cb5a14996b8ff3 P cae774377c4242fb87db8fa139f33c55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ae07d49ebd941b2bf8e3fde33a76b6f in term 1.
I20260808 03:51:18.131106 29927 leader_election.cc:302] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ae07d49ebd941b2bf8e3fde33a76b6f, cae774377c4242fb87db8fa139f33c55; no voters:
I20260808 03:51:18.131381 30691 raft_consensus.cc:2781] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:18.131844 30691 raft_consensus.cc:686] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 1 LEADER]: Becoming Leader. State: Replica: 6ae07d49ebd941b2bf8e3fde33a76b6f, State: Running, Role: LEADER
I20260808 03:51:18.132263 30691 consensus_queue.cc:237] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:18.135769 29797 catalog_manager.cc:5495] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ae07d49ebd941b2bf8e3fde33a76b6f (127.26.146.129). New cstate: current_term: 1 leader_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } health_report { overall_health: UNKNOWN } } }
I20260808 03:51:18.161896 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:18.155899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46476 (local address 127.26.146.129:37659)
0808 03:51:18.156495 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:18.156499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:18.156516 (+ 17us) server_negotiation.cc:407] Connection header received
0808 03:51:18.156600 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:18.156603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:18.156690 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:18.156857 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:51:18.157295 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:18.158207 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:18.159644 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:18.160078 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:18.160211 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:18.160690 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:51:18.161258 (+ 568us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:51:18.161388 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:18.161506 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 03:51:18.161586 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":344,"threads_started":1}
I20260808 03:51:18.175791 30695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:18.170703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:59689 (local address 127.26.146.130:33031)
0808 03:51:18.170862 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:18.170866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:18.171717 (+ 851us) server_negotiation.cc:407] Connection header received
0808 03:51:18.171900 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:18.171904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:18.171967 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:18.172031 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:51:18.172423 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:18.173095 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:18.174416 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:18.174797 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:18.174907 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:18.175261 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:51:18.175375 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:51:18.175437 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:18.175534 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 03:51:18.175597 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 03:51:18.176011 30705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:18.170563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:33031 (local address 127.26.146.129:59689)
0808 03:51:18.171581 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:18.171597 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:18.171773 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:18.172048 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:18.172051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:18.172060 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 03:51:18.172318 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:18.172327 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:18.173211 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:18.173215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:18.174277 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:18.174288 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:18.174941 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:18.174945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:18.175071 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:18.175123 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 03:51:18.175450 (+ 327us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 03:51:18.175458 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 03:51:18.175513 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 03:51:18.175652 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":870,"thread_start_us":599,"threads_started":1}
I20260808 03:51:18.184141 30258 raft_consensus.cc:1261] T fb14261ada0949f6a7cb5a14996b8ff3 P cae774377c4242fb87db8fa139f33c55 [term 1 FOLLOWER]: Refusing update from remote peer 6ae07d49ebd941b2bf8e3fde33a76b6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:51:18.184123 30452 raft_consensus.cc:1261] T fb14261ada0949f6a7cb5a14996b8ff3 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 1 FOLLOWER]: Refusing update from remote peer 6ae07d49ebd941b2bf8e3fde33a76b6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:51:18.184764 30697 consensus_queue.cc:1035] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:51:18.185271 30691 consensus_queue.cc:1035] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:51:18.212481 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:18.206517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:56123 (local address 127.26.146.129:37659)
0808 03:51:18.206922 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:18.206929 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:18.207593 (+ 664us) server_negotiation.cc:407] Connection header received
0808 03:51:18.207781 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:18.207787 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:18.207898 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:18.207964 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:51:18.208556 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:18.209387 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:18.210899 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:18.211296 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:18.211404 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:18.211795 (+ 391us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:51:18.211937 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:51:18.212012 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:18.212145 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 03:51:18.212219 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277}
I20260808 03:51:18.212563 30711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:18.206549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:37659 (local address 127.26.146.130:56123)
0808 03:51:18.207459 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:18.207484 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:18.207661 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:18.207995 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:18.208000 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:18.208012 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 03:51:18.208363 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:18.208374 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:18.209505 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:18.209511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:18.210760 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:18.210773 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:18.211461 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:18.211469 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:18.211608 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:18.211647 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 03:51:18.212026 (+ 379us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 03:51:18.212038 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 03:51:18.212113 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 03:51:18.212254 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"thread_start_us":577,"threads_started":1}
I20260808 03:51:18.239776 29797 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47284:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:51:18.240083 29797 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47284:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:51:18.243320 29797 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e: Sending DeleteTablet for 3 replicas of tablet fb14261ada0949f6a7cb5a14996b8ff3
I20260808 03:51:18.244091 29995 tablet_service.cc:1514] Processing DeleteTablet for tablet fb14261ada0949f6a7cb5a14996b8ff3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:18 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:46470
I20260808 03:51:18.244148 30215 tablet_service.cc:1514] Processing DeleteTablet for tablet fb14261ada0949f6a7cb5a14996b8ff3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:18 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:37352
I20260808 03:51:18.244406 30669 tablet_replica.cc:330] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f: stopping tablet replica
I20260808 03:51:18.244369 30425 tablet_service.cc:1514] Processing DeleteTablet for tablet fb14261ada0949f6a7cb5a14996b8ff3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:18 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:56228
I20260808 03:51:18.244437 30670 tablet_replica.cc:330] T fb14261ada0949f6a7cb5a14996b8ff3 P cae774377c4242fb87db8fa139f33c55: stopping tablet replica
I20260808 03:51:18.244607 30670 raft_consensus.cc:2229] T fb14261ada0949f6a7cb5a14996b8ff3 P cae774377c4242fb87db8fa139f33c55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:51:18.244616 30669 raft_consensus.cc:2229] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:51:18.244761 30670 raft_consensus.cc:2258] T fb14261ada0949f6a7cb5a14996b8ff3 P cae774377c4242fb87db8fa139f33c55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:18.244992 30668 tablet_replica.cc:330] T fb14261ada0949f6a7cb5a14996b8ff3 P 5c3f01dda9234f2aa081ad4435bf94a1: stopping tablet replica
I20260808 03:51:18.245093 30669 raft_consensus.cc:2258] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:18.245157 30668 raft_consensus.cc:2229] T fb14261ada0949f6a7cb5a14996b8ff3 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:51:18.245304 30668 raft_consensus.cc:2258] T fb14261ada0949f6a7cb5a14996b8ff3 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:18.246439 30668 ts_tablet_manager.cc:1905] T fb14261ada0949f6a7cb5a14996b8ff3 P 5c3f01dda9234f2aa081ad4435bf94a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:18.246439 30670 ts_tablet_manager.cc:1905] T fb14261ada0949f6a7cb5a14996b8ff3 P cae774377c4242fb87db8fa139f33c55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:18.247536 30669 ts_tablet_manager.cc:1905] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:18.250008 29797 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47284:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:51:18.250594 29797 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:51:18.252879 30668 ts_tablet_manager.cc:1918] T fb14261ada0949f6a7cb5a14996b8ff3 P 5c3f01dda9234f2aa081ad4435bf94a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 03:51:18.252879 30670 ts_tablet_manager.cc:1918] T fb14261ada0949f6a7cb5a14996b8ff3 P cae774377c4242fb87db8fa139f33c55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 03:51:18.253031 30668 log.cc:1196] T fb14261ada0949f6a7cb5a14996b8ff3 P 5c3f01dda9234f2aa081ad4435bf94a1: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/fb14261ada0949f6a7cb5a14996b8ff3
I20260808 03:51:18.253031 30670 log.cc:1196] T fb14261ada0949f6a7cb5a14996b8ff3 P cae774377c4242fb87db8fa139f33c55: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/fb14261ada0949f6a7cb5a14996b8ff3
I20260808 03:51:18.253455 30668 ts_tablet_manager.cc:1939] T fb14261ada0949f6a7cb5a14996b8ff3 P 5c3f01dda9234f2aa081ad4435bf94a1: Deleting consensus metadata
I20260808 03:51:18.253459 30670 ts_tablet_manager.cc:1939] T fb14261ada0949f6a7cb5a14996b8ff3 P cae774377c4242fb87db8fa139f33c55: Deleting consensus metadata
I20260808 03:51:18.254546 29777 catalog_manager.cc:4841] TS 5c3f01dda9234f2aa081ad4435bf94a1 (127.26.146.131:36241): tablet fb14261ada0949f6a7cb5a14996b8ff3 (table test-table [id=ea870b5ba0ef4fa282effe881a13e12d]) successfully deleted
I20260808 03:51:18.256197 30669 ts_tablet_manager.cc:1918] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 03:51:18.256366 30669 log.cc:1196] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-0/wal/wals/fb14261ada0949f6a7cb5a14996b8ff3
I20260808 03:51:18.256789 30669 ts_tablet_manager.cc:1939] T fb14261ada0949f6a7cb5a14996b8ff3 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Deleting consensus metadata
I20260808 03:51:18.257010 29775 catalog_manager.cc:4841] TS cae774377c4242fb87db8fa139f33c55 (127.26.146.130:33031): tablet fb14261ada0949f6a7cb5a14996b8ff3 (table test-table [id=ea870b5ba0ef4fa282effe881a13e12d]) successfully deleted
I20260808 03:51:18.258229 29779 catalog_manager.cc:4841] TS 6ae07d49ebd941b2bf8e3fde33a76b6f (127.26.146.129:37659): tablet fb14261ada0949f6a7cb5a14996b8ff3 (table test-table [id=ea870b5ba0ef4fa282effe881a13e12d]) successfully deleted
I20260808 03:51:18.258656 29995 tablet_service.cc:1467] Processing CreateTablet for tablet 57fb9e9e40784385a097aa6f9a455ba2 (DEFAULT_TABLE table=test-table [id=05099f2f22e34e2b83b2cdec2ecae9ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:18.258852 30425 tablet_service.cc:1467] Processing CreateTablet for tablet 57fb9e9e40784385a097aa6f9a455ba2 (DEFAULT_TABLE table=test-table [id=05099f2f22e34e2b83b2cdec2ecae9ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:18.258909 30215 tablet_service.cc:1467] Processing CreateTablet for tablet 57fb9e9e40784385a097aa6f9a455ba2 (DEFAULT_TABLE table=test-table [id=05099f2f22e34e2b83b2cdec2ecae9ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:18.259085 29995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57fb9e9e40784385a097aa6f9a455ba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:18.259261 30425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57fb9e9e40784385a097aa6f9a455ba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:18.259332 30215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57fb9e9e40784385a097aa6f9a455ba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:18.265625 30687 tablet_bootstrap.cc:492] T 57fb9e9e40784385a097aa6f9a455ba2 P cae774377c4242fb87db8fa139f33c55: Bootstrap starting.
I20260808 03:51:18.266032 30685 tablet_bootstrap.cc:492] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1: Bootstrap starting.
I20260808 03:51:18.266224 30683 tablet_bootstrap.cc:492] T 57fb9e9e40784385a097aa6f9a455ba2 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Bootstrap starting.
I20260808 03:51:18.267292 30687 tablet_bootstrap.cc:654] T 57fb9e9e40784385a097aa6f9a455ba2 P cae774377c4242fb87db8fa139f33c55: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:18.267633 30685 tablet_bootstrap.cc:654] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:18.267884 30683 tablet_bootstrap.cc:654] T 57fb9e9e40784385a097aa6f9a455ba2 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:18.277246 30683 tablet_bootstrap.cc:492] T 57fb9e9e40784385a097aa6f9a455ba2 P 6ae07d49ebd941b2bf8e3fde33a76b6f: No bootstrap required, opened a new log
I20260808 03:51:18.277442 30683 ts_tablet_manager.cc:1397] T 57fb9e9e40784385a097aa6f9a455ba2 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.008s
I20260808 03:51:18.278262 30683 raft_consensus.cc:348] T 57fb9e9e40784385a097aa6f9a455ba2 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:18.278544 30683 raft_consensus.cc:374] T 57fb9e9e40784385a097aa6f9a455ba2 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:18.278635 30683 raft_consensus.cc:729] T 57fb9e9e40784385a097aa6f9a455ba2 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ae07d49ebd941b2bf8e3fde33a76b6f, State: Initialized, Role: FOLLOWER
I20260808 03:51:18.278936 30683 consensus_queue.cc:260] T 57fb9e9e40784385a097aa6f9a455ba2 P 6ae07d49ebd941b2bf8e3fde33a76b6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:18.280665 30683 ts_tablet_manager.cc:1428] T 57fb9e9e40784385a097aa6f9a455ba2 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260808 03:51:18.282459 30685 tablet_bootstrap.cc:492] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1: No bootstrap required, opened a new log
I20260808 03:51:18.282687 30685 ts_tablet_manager.cc:1397] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.008s
I20260808 03:51:18.282788 30687 tablet_bootstrap.cc:492] T 57fb9e9e40784385a097aa6f9a455ba2 P cae774377c4242fb87db8fa139f33c55: No bootstrap required, opened a new log
I20260808 03:51:18.283046 30687 ts_tablet_manager.cc:1397] T 57fb9e9e40784385a097aa6f9a455ba2 P cae774377c4242fb87db8fa139f33c55: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.005s
I20260808 03:51:18.283655 30685 raft_consensus.cc:348] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:18.284001 30685 raft_consensus.cc:374] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:18.283954 30687 raft_consensus.cc:348] T 57fb9e9e40784385a097aa6f9a455ba2 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:18.284195 30687 raft_consensus.cc:374] T 57fb9e9e40784385a097aa6f9a455ba2 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:18.284116 30685 raft_consensus.cc:729] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3f01dda9234f2aa081ad4435bf94a1, State: Initialized, Role: FOLLOWER
I20260808 03:51:18.284272 30687 raft_consensus.cc:729] T 57fb9e9e40784385a097aa6f9a455ba2 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cae774377c4242fb87db8fa139f33c55, State: Initialized, Role: FOLLOWER
I20260808 03:51:18.284505 30687 consensus_queue.cc:260] T 57fb9e9e40784385a097aa6f9a455ba2 P cae774377c4242fb87db8fa139f33c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:18.284574 30685 consensus_queue.cc:260] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:18.285058 30687 ts_tablet_manager.cc:1428] T 57fb9e9e40784385a097aa6f9a455ba2 P cae774377c4242fb87db8fa139f33c55: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 03:51:18.285884 30685 ts_tablet_manager.cc:1428] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 03:51:18.455423 30655 raft_consensus.cc:482] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:51:18.455652 30655 raft_consensus.cc:504] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:18.456508 30655 leader_election.cc:288] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ae07d49ebd941b2bf8e3fde33a76b6f (127.26.146.129:37659), cae774377c4242fb87db8fa139f33c55 (127.26.146.130:33031)
I20260808 03:51:18.457170 30258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57fb9e9e40784385a097aa6f9a455ba2" candidate_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cae774377c4242fb87db8fa139f33c55" is_pre_election: true
I20260808 03:51:18.457125 30029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57fb9e9e40784385a097aa6f9a455ba2" candidate_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" is_pre_election: true
I20260808 03:51:18.457453 30258 raft_consensus.cc:2454] T 57fb9e9e40784385a097aa6f9a455ba2 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3f01dda9234f2aa081ad4435bf94a1 in term 0.
I20260808 03:51:18.457466 30029 raft_consensus.cc:2454] T 57fb9e9e40784385a097aa6f9a455ba2 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3f01dda9234f2aa081ad4435bf94a1 in term 0.
I20260808 03:51:18.458014 30353 leader_election.cc:302] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3f01dda9234f2aa081ad4435bf94a1, cae774377c4242fb87db8fa139f33c55; no voters:
I20260808 03:51:18.458330 30655 raft_consensus.cc:2781] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:51:18.458456 30655 raft_consensus.cc:482] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:51:18.458539 30655 raft_consensus.cc:3037] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:18.460896 30655 raft_consensus.cc:504] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:18.461364 30655 leader_election.cc:288] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1 [CANDIDATE]: Term 1 election: Requested vote from peers 6ae07d49ebd941b2bf8e3fde33a76b6f (127.26.146.129:37659), cae774377c4242fb87db8fa139f33c55 (127.26.146.130:33031)
I20260808 03:51:18.461892 30029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57fb9e9e40784385a097aa6f9a455ba2" candidate_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f"
I20260808 03:51:18.461926 30258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57fb9e9e40784385a097aa6f9a455ba2" candidate_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cae774377c4242fb87db8fa139f33c55"
I20260808 03:51:18.462070 30029 raft_consensus.cc:3037] T 57fb9e9e40784385a097aa6f9a455ba2 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:18.462146 30258 raft_consensus.cc:3037] T 57fb9e9e40784385a097aa6f9a455ba2 P cae774377c4242fb87db8fa139f33c55 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:18.464581 30029 raft_consensus.cc:2454] T 57fb9e9e40784385a097aa6f9a455ba2 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3f01dda9234f2aa081ad4435bf94a1 in term 1.
I20260808 03:51:18.465081 30258 raft_consensus.cc:2454] T 57fb9e9e40784385a097aa6f9a455ba2 P cae774377c4242fb87db8fa139f33c55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3f01dda9234f2aa081ad4435bf94a1 in term 1.
I20260808 03:51:18.465500 30353 leader_election.cc:302] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3f01dda9234f2aa081ad4435bf94a1, cae774377c4242fb87db8fa139f33c55; no voters:
I20260808 03:51:18.465766 30655 raft_consensus.cc:2781] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:18.465926 30655 raft_consensus.cc:686] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3f01dda9234f2aa081ad4435bf94a1, State: Running, Role: LEADER
I20260808 03:51:18.466187 30655 consensus_queue.cc:237] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } }
I20260808 03:51:18.468815 29797 catalog_manager.cc:5495] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3f01dda9234f2aa081ad4435bf94a1 (127.26.146.131). New cstate: current_term: 1 leader_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c3f01dda9234f2aa081ad4435bf94a1" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 36241 } health_report { overall_health: HEALTHY } } }
I20260808 03:51:18.522796 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:18.517356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56238 (local address 127.26.146.131:36241)
0808 03:51:18.517529 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:18.517535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:18.517756 (+ 221us) server_negotiation.cc:407] Connection header received
0808 03:51:18.517936 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:18.517940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:18.518005 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:18.518073 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:51:18.518507 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:18.519233 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:18.521118 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:18.521538 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:18.521635 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:18.522109 (+ 474us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:51:18.522215 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:51:18.522273 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:18.522433 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 03:51:18.522512 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 03:51:18.532354 30029 raft_consensus.cc:1261] T 57fb9e9e40784385a097aa6f9a455ba2 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 1 FOLLOWER]: Refusing update from remote peer 5c3f01dda9234f2aa081ad4435bf94a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:51:18.532454 30258 raft_consensus.cc:1261] T 57fb9e9e40784385a097aa6f9a455ba2 P cae774377c4242fb87db8fa139f33c55 [term 1 FOLLOWER]: Refusing update from remote peer 5c3f01dda9234f2aa081ad4435bf94a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:51:18.532919 30637 consensus_queue.cc:1035] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ae07d49ebd941b2bf8e3fde33a76b6f" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:51:18.533540 30655 consensus_queue.cc:1035] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cae774377c4242fb87db8fa139f33c55" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 33031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 03:51:18.563311 30641 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 03:51:20.157902 29797 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47284:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:51:20.158123 29797 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47284:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:51:20.161865 29797 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 02e199820a3c4e39b4da67726453221e: Sending DeleteTablet for 3 replicas of tablet 57fb9e9e40784385a097aa6f9a455ba2
I20260808 03:51:20.162545 29995 tablet_service.cc:1514] Processing DeleteTablet for tablet 57fb9e9e40784385a097aa6f9a455ba2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:20 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:46470
I20260808 03:51:20.162686 30215 tablet_service.cc:1514] Processing DeleteTablet for tablet 57fb9e9e40784385a097aa6f9a455ba2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:20 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:37352
I20260808 03:51:20.163275 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 29903
I20260808 03:51:20.163264 30425 tablet_service.cc:1514] Processing DeleteTablet for tablet 57fb9e9e40784385a097aa6f9a455ba2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:20 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:56228
I20260808 03:51:20.163502 30756 tablet_replica.cc:330] T 57fb9e9e40784385a097aa6f9a455ba2 P cae774377c4242fb87db8fa139f33c55: stopping tablet replica
I20260808 03:51:20.163761 30756 raft_consensus.cc:2229] T 57fb9e9e40784385a097aa6f9a455ba2 P cae774377c4242fb87db8fa139f33c55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:51:20.164136 30756 raft_consensus.cc:2258] T 57fb9e9e40784385a097aa6f9a455ba2 P cae774377c4242fb87db8fa139f33c55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:20.165126 30756 ts_tablet_manager.cc:1905] T 57fb9e9e40784385a097aa6f9a455ba2 P cae774377c4242fb87db8fa139f33c55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:20.172408 30756 ts_tablet_manager.cc:1918] T 57fb9e9e40784385a097aa6f9a455ba2 P cae774377c4242fb87db8fa139f33c55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 03:51:20.172657 30756 log.cc:1196] T 57fb9e9e40784385a097aa6f9a455ba2 P cae774377c4242fb87db8fa139f33c55: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/57fb9e9e40784385a097aa6f9a455ba2
I20260808 03:51:20.173226 30756 ts_tablet_manager.cc:1939] T 57fb9e9e40784385a097aa6f9a455ba2 P cae774377c4242fb87db8fa139f33c55: Deleting consensus metadata
I20260808 03:51:20.174448 29775 catalog_manager.cc:4841] TS cae774377c4242fb87db8fa139f33c55 (127.26.146.130:33031): tablet 57fb9e9e40784385a097aa6f9a455ba2 (table test-table [id=05099f2f22e34e2b83b2cdec2ecae9ca]) successfully deleted
I20260808 03:51:20.175714 30757 tablet_replica.cc:330] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1: stopping tablet replica
I20260808 03:51:20.176003 30757 raft_consensus.cc:2229] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:51:20.176378 30757 raft_consensus.cc:2258] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:20.177381 30757 ts_tablet_manager.cc:1905] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:20.181237 30755 tablet_replica.cc:330] T 57fb9e9e40784385a097aa6f9a455ba2 P 6ae07d49ebd941b2bf8e3fde33a76b6f: stopping tablet replica
I20260808 03:51:20.181493 30755 raft_consensus.cc:2229] T 57fb9e9e40784385a097aa6f9a455ba2 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:51:20.181749 30755 raft_consensus.cc:2258] T 57fb9e9e40784385a097aa6f9a455ba2 P 6ae07d49ebd941b2bf8e3fde33a76b6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:20.182729 30755 ts_tablet_manager.cc:1905] T 57fb9e9e40784385a097aa6f9a455ba2 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:20.183499 30757 ts_tablet_manager.cc:1918] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 03:51:20.183636 30757 log.cc:1196] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/57fb9e9e40784385a097aa6f9a455ba2
I20260808 03:51:20.184084 30757 ts_tablet_manager.cc:1939] T 57fb9e9e40784385a097aa6f9a455ba2 P 5c3f01dda9234f2aa081ad4435bf94a1: Deleting consensus metadata
I20260808 03:51:20.185007 29777 catalog_manager.cc:4841] TS 5c3f01dda9234f2aa081ad4435bf94a1 (127.26.146.131:36241): tablet 57fb9e9e40784385a097aa6f9a455ba2 (table test-table [id=05099f2f22e34e2b83b2cdec2ecae9ca]) successfully deleted
I20260808 03:51:20.188293 30755 ts_tablet_manager.cc:1918] T 57fb9e9e40784385a097aa6f9a455ba2 P 6ae07d49ebd941b2bf8e3fde33a76b6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 03:51:20.188401 30755 log.cc:1196] T 57fb9e9e40784385a097aa6f9a455ba2 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786161058098901-27210-0/minicluster-data/ts-0/wal/wals/57fb9e9e40784385a097aa6f9a455ba2
I20260808 03:51:20.188723 30755 ts_tablet_manager.cc:1939] T 57fb9e9e40784385a097aa6f9a455ba2 P 6ae07d49ebd941b2bf8e3fde33a76b6f: Deleting consensus metadata
I20260808 03:51:20.189528 29779 catalog_manager.cc:4841] TS 6ae07d49ebd941b2bf8e3fde33a76b6f (127.26.146.129:37659): tablet 57fb9e9e40784385a097aa6f9a455ba2 (table test-table [id=05099f2f22e34e2b83b2cdec2ecae9ca]) successfully deleted
Aug 08 03:51:20 dist-test-slave-jlzn krb5kdc[29731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161078, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.146.129@KRBTEST.COM
I20260808 03:51:20.198779 30760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:20.165064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46486 (local address 127.26.146.129:37659)
0808 03:51:20.180054 (+ 14990us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:20.180061 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:20.180588 (+ 527us) server_negotiation.cc:407] Connection header received
0808 03:51:20.180900 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:20.180906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:20.180997 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:20.181146 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:20.182667 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:20.183882 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:20.185611 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:20.186142 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:20.186243 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:20.193954 (+ 7711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:20.193969 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:20.193986 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:20.194024 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:20.196610 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:20.197178 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:20.197185 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:20.197187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:20.197320 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:20.197587 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:20.197590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:20.197592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:20.197926 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:20.198158 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:20.198351 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 03:51:20.198516 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":14821,"thread_start_us":14661,"threads_started":1}
W20260808 03:51:20.199405 30097 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46486
W20260808 03:51:20.200027 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 29903: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:51:20.200198 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 29903
I20260808 03:51:20.235925 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 30112
Aug 08 03:51:20 dist-test-slave-jlzn krb5kdc[29731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161078, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.146.130@KRBTEST.COM
I20260808 03:51:20.249791 30761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:20.236694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37378 (local address 127.26.146.130:33031)
0808 03:51:20.237214 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:20.237219 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:20.237235 (+ 16us) server_negotiation.cc:407] Connection header received
0808 03:51:20.237291 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:20.237295 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:20.237359 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:20.237446 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:20.238581 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:20.239267 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:20.240489 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:20.240918 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:20.241010 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:20.244887 (+ 3877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:20.244900 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:20.244916 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:20.244947 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:20.247471 (+ 2524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:20.248161 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:20.248168 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:20.248170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:20.248293 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:20.248583 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:20.248586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:20.248588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:20.248934 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:20.249159 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:20.249364 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 03:51:20.249535 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":263,"threads_started":1}
W20260808 03:51:20.250412 30317 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37378
W20260808 03:51:20.250999 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 30112: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:51:20.251160 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 30112
I20260808 03:51:20.285862 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 30331
Aug 08 03:51:20 dist-test-slave-jlzn krb5kdc[29731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161078, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.146.131@KRBTEST.COM
I20260808 03:51:20.300545 30763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:20.286535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56248 (local address 127.26.146.131:36241)
0808 03:51:20.287050 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:20.287054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:20.287068 (+ 14us) server_negotiation.cc:407] Connection header received
0808 03:51:20.287123 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:20.287128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:20.287189 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:20.287280 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:20.288662 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:20.289360 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:20.290592 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:20.291006 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:20.291114 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:20.295215 (+ 4101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:20.295230 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:20.295248 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:20.295285 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:20.297836 (+ 2551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:20.298482 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:20.298489 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:20.298492 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:20.298630 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:20.298998 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:20.299002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:20.299004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:20.299360 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:20.299648 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:20.300057 (+ 409us) server_negotiation.cc:299] Negotiation successful
0808 03:51:20.300268 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":260,"threads_started":1}
W20260808 03:51:20.301182 30511 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56248
W20260808 03:51:20.301765 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 30331: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:51:20.301900 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 30331
I20260808 03:51:20.328649 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 29749
I20260808 03:51:20.329840 29855 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:20.445886 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 29749
2026-08-08T03:51:20Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8260 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:51:20 dist-test-slave-jlzn krb5kdc[30767](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:51:20 dist-test-slave-jlzn krb5kdc[30767](info): set up 2 sockets
Aug 08 03:51:20 dist-test-slave-jlzn krb5kdc[30767](info): commencing operation
krb5kdc: starting...
W20260808 03:51:21.622292 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:51:21 dist-test-slave-jlzn krb5kdc[30767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:51:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:51:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:51:21.818976 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:35137
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:38829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:35137
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:22.142190 30783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:22.142498 30783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:22.142652 30783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:22.149746 30783 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:51:22.149865 30783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:22.149946 30783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:22.150023 30783 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:51:22.150089 30783 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:51:22.159839 30783 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:38829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:35137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:35137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:22.161141 30783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:22.162472 30783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:22.174155 30789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:22.174132 30794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:22.174201 30792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:22.174477 30783 server_base.cc:1034] running on GCE node
I20260808 03:51:22.175369 30783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:22.176753 30783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:22.177963 30783 hybrid_clock.cc:648] HybridClock initialized: now 1786161082177910 us; error 38 us; skew 500 ppm
I20260808 03:51:22.178285 30783 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:22 dist-test-slave-jlzn krb5kdc[30767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161082, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:22.181345 30783 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:51:22.182714 30783 webserver.cc:460] Webserver started at http://127.26.146.190:34727/ using document root <none> and password file <none>
I20260808 03:51:22.183362 30783 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:22.183477 30783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:22.183776 30783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:22.186691 30783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "a258729074474ff0b6ac857502d79fe1"
format_stamp: "Formatted at 2026-08-08 03:51:22 on dist-test-slave-jlzn"
server_key: "00326b343cb2af12a081438ba459c848"
server_key_iv: "43630487d1bbafd920804a09c9b9fbfe"
server_key_version: "encryptionkey@0"
I20260808 03:51:22.187428 30783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "a258729074474ff0b6ac857502d79fe1"
format_stamp: "Formatted at 2026-08-08 03:51:22 on dist-test-slave-jlzn"
server_key: "00326b343cb2af12a081438ba459c848"
server_key_iv: "43630487d1bbafd920804a09c9b9fbfe"
server_key_version: "encryptionkey@0"
I20260808 03:51:22.192431 30783 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260808 03:51:22.195705 30807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:22.197003 30783 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 03:51:22.197172 30783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "a258729074474ff0b6ac857502d79fe1"
format_stamp: "Formatted at 2026-08-08 03:51:22 on dist-test-slave-jlzn"
server_key: "00326b343cb2af12a081438ba459c848"
server_key_iv: "43630487d1bbafd920804a09c9b9fbfe"
server_key_version: "encryptionkey@0"
I20260808 03:51:22.197353 30783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:22.207576 30783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:22.211557 30783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:22.211829 30783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:22.227891 30783 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:35137
I20260808 03:51:22.227922 30881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:35137 every 8 connection(s)
I20260808 03:51:22.229269 30783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:51:22.232896 30884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:22.237162 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 30783
I20260808 03:51:22.237354 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:51:22.237980 27210 external_mini_cluster.cc:1428] Setting key 2a18411e169885388aab69a18e73e262
I20260808 03:51:22.243495 30884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a258729074474ff0b6ac857502d79fe1: Bootstrap starting.
I20260808 03:51:22.246546 30884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a258729074474ff0b6ac857502d79fe1: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:22.247534 30884 log.cc:824] T 00000000000000000000000000000000 P a258729074474ff0b6ac857502d79fe1: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:22.250862 30884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a258729074474ff0b6ac857502d79fe1: No bootstrap required, opened a new log
Aug 08 03:51:22 dist-test-slave-jlzn krb5kdc[30767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:22.258467 30884 raft_consensus.cc:348] T 00000000000000000000000000000000 P a258729074474ff0b6ac857502d79fe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a258729074474ff0b6ac857502d79fe1" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 35137 } }
I20260808 03:51:22.258888 30884 raft_consensus.cc:374] T 00000000000000000000000000000000 P a258729074474ff0b6ac857502d79fe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:22.259002 30884 raft_consensus.cc:729] T 00000000000000000000000000000000 P a258729074474ff0b6ac857502d79fe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a258729074474ff0b6ac857502d79fe1, State: Initialized, Role: FOLLOWER
I20260808 03:51:22.259615 30884 consensus_queue.cc:260] T 00000000000000000000000000000000 P a258729074474ff0b6ac857502d79fe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a258729074474ff0b6ac857502d79fe1" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 35137 } }
I20260808 03:51:22.259747 30887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:22.239891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46206 (local address 127.26.146.190:35137)
0808 03:51:22.240667 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:22.240679 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:22.240731 (+ 52us) server_negotiation.cc:407] Connection header received
0808 03:51:22.242443 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:22.242466 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:22.242822 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:22.243178 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:22.244655 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:22.245975 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:22.247296 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:22.248099 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:22.248304 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:22.252738 (+ 4434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:22.252746 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:22.252761 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:22.252798 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:22.255846 (+ 3048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:22.256495 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:22.256501 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:22.256507 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:22.256677 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:22.257052 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:22.257055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:22.257057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:22.257586 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:22.257881 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:22.258279 (+ 398us) server_negotiation.cc:299] Negotiation successful
0808 03:51:22.258688 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":365,"threads_started":1}
I20260808 03:51:22.259884 30884 raft_consensus.cc:388] T 00000000000000000000000000000000 P a258729074474ff0b6ac857502d79fe1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:51:22.260027 30884 raft_consensus.cc:482] T 00000000000000000000000000000000 P a258729074474ff0b6ac857502d79fe1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:51:22.260187 30884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a258729074474ff0b6ac857502d79fe1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:22.263031 30884 raft_consensus.cc:504] T 00000000000000000000000000000000 P a258729074474ff0b6ac857502d79fe1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a258729074474ff0b6ac857502d79fe1" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 35137 } }
I20260808 03:51:22.263468 30884 leader_election.cc:302] T 00000000000000000000000000000000 P a258729074474ff0b6ac857502d79fe1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a258729074474ff0b6ac857502d79fe1; no voters:
I20260808 03:51:22.264002 30884 leader_election.cc:288] T 00000000000000000000000000000000 P a258729074474ff0b6ac857502d79fe1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:51:22.264101 30891 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a258729074474ff0b6ac857502d79fe1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:22.264755 30891 raft_consensus.cc:686] T 00000000000000000000000000000000 P a258729074474ff0b6ac857502d79fe1 [term 1 LEADER]: Becoming Leader. State: Replica: a258729074474ff0b6ac857502d79fe1, State: Running, Role: LEADER
I20260808 03:51:22.265211 30891 consensus_queue.cc:237] T 00000000000000000000000000000000 P a258729074474ff0b6ac857502d79fe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a258729074474ff0b6ac857502d79fe1" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 35137 } }
I20260808 03:51:22.266057 30884 sys_catalog.cc:564] T 00000000000000000000000000000000 P a258729074474ff0b6ac857502d79fe1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:51:22.267906 30892 sys_catalog.cc:455] T 00000000000000000000000000000000 P a258729074474ff0b6ac857502d79fe1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a258729074474ff0b6ac857502d79fe1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a258729074474ff0b6ac857502d79fe1" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 35137 } } }
I20260808 03:51:22.268409 30892 sys_catalog.cc:458] T 00000000000000000000000000000000 P a258729074474ff0b6ac857502d79fe1 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:22.267931 30893 sys_catalog.cc:455] T 00000000000000000000000000000000 P a258729074474ff0b6ac857502d79fe1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a258729074474ff0b6ac857502d79fe1. Latest consensus state: current_term: 1 leader_uuid: "a258729074474ff0b6ac857502d79fe1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a258729074474ff0b6ac857502d79fe1" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 35137 } } }
I20260808 03:51:22.269753 30893 sys_catalog.cc:458] T 00000000000000000000000000000000 P a258729074474ff0b6ac857502d79fe1 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:22.271230 30900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:51:22.276197 30900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:51:22.283917 30900 catalog_manager.cc:1347] Generated new cluster ID: bd4a7f46bc344e51822f625f38cd9b12
I20260808 03:51:22.284027 30900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:51:22.297324 30900 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:51:22.298545 30900 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:51:22.312093 30900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a258729074474ff0b6ac857502d79fe1: Generated new TSK 0
I20260808 03:51:22.312870 30900 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
I20260808 03:51:22.369221 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:35137
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:38829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:22.690696 30926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:22.690963 30926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:22.691232 30926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:22.698418 30926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:22.698524 30926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:22.698679 30926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:51:22.708357 30926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:38829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:35137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:22.709591 30926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:22.710907 30926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:22.724143 30934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:22.724185 30937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:22.724100 30935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:22.729238 30926 server_base.cc:1034] running on GCE node
I20260808 03:51:22.729892 30926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:22.730822 30926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:22.732012 30926 hybrid_clock.cc:648] HybridClock initialized: now 1786161082731968 us; error 37 us; skew 500 ppm
I20260808 03:51:22.732316 30926 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:22 dist-test-slave-jlzn krb5kdc[30767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161082, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:22.735302 30926 init.cc:377] Logged in from keytab as kudu/127.26.146.129@KRBTEST.COM (short username kudu)
I20260808 03:51:22.736750 30926 webserver.cc:460] Webserver started at http://127.26.146.129:41225/ using document root <none> and password file <none>
I20260808 03:51:22.737452 30926 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:22.737568 30926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:22.737857 30926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:22.740816 30926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "1c5b3a331d094fc380042a80e3bde1b3"
format_stamp: "Formatted at 2026-08-08 03:51:22 on dist-test-slave-jlzn"
server_key: "ce993f8b328849ebee7888d602ed1eec"
server_key_iv: "8b79765dd0c058c3c607ae2f04d1b96b"
server_key_version: "encryptionkey@0"
I20260808 03:51:22.741575 30926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "1c5b3a331d094fc380042a80e3bde1b3"
format_stamp: "Formatted at 2026-08-08 03:51:22 on dist-test-slave-jlzn"
server_key: "ce993f8b328849ebee7888d602ed1eec"
server_key_iv: "8b79765dd0c058c3c607ae2f04d1b96b"
server_key_version: "encryptionkey@0"
I20260808 03:51:22.746383 30926 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 03:51:22.749600 30947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:22.750844 30926 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 03:51:22.750998 30926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "1c5b3a331d094fc380042a80e3bde1b3"
format_stamp: "Formatted at 2026-08-08 03:51:22 on dist-test-slave-jlzn"
server_key: "ce993f8b328849ebee7888d602ed1eec"
server_key_iv: "8b79765dd0c058c3c607ae2f04d1b96b"
server_key_version: "encryptionkey@0"
I20260808 03:51:22.751174 30926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:22.766705 30926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:22.770520 30926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:22.770779 30926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:22.771656 30926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:22.772917 30926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:22.772989 30926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:22.773108 30926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:22.773175 30926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:22.800967 30926 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:46455
I20260808 03:51:22.801012 31116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:46455 every 8 connection(s)
I20260808 03:51:22.802263 30926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:51:22.808249 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 30926
I20260808 03:51:22.808425 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:51:22.809062 27210 external_mini_cluster.cc:1428] Setting key e4b315a118a263c1c452a2fc28c734c6
Aug 08 03:51:22 dist-test-slave-jlzn krb5kdc[30767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161082, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:22.821805 31121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:22.804781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:48373 (local address 127.26.146.190:35137)
0808 03:51:22.805188 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:22.805192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:22.806154 (+ 962us) server_negotiation.cc:407] Connection header received
0808 03:51:22.807930 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:22.807936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:22.808029 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:22.808126 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:22.809662 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:22.810506 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:22.812284 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:22.812711 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:22.812820 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:22.816197 (+ 3377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:22.816202 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:22.816205 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:22.816237 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:22.818830 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:22.819538 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:22.819544 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:22.819547 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:22.819684 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:22.820114 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:22.820119 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:22.820123 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:22.820460 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:22.820602 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:22.821230 (+ 628us) server_negotiation.cc:299] Negotiation successful
0808 03:51:22.821437 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":220,"threads_started":1}
I20260808 03:51:22.822606 31122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:22.805218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:35137 (local address 127.26.146.129:48373)
0808 03:51:22.805982 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:22.806040 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:22.807663 (+ 1623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:22.808380 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:22.808391 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:22.808918 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:22.809498 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:22.809517 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:22.810667 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:22.810671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:22.812138 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:22.812151 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:22.812923 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:22.812931 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:22.813150 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:22.813773 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:22.813795 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:22.815983 (+ 2188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:22.818984 (+ 3001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:22.818990 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:22.819001 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:22.819408 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:22.819779 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:22.819782 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:22.819785 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:22.820001 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:22.820564 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:22.820569 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:22.820883 (+ 314us) client_negotiation.cc:769] Sending connection context
0808 03:51:22.821098 (+ 215us) client_negotiation.cc:240] Negotiation successful
0808 03:51:22.821463 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":279,"threads_started":1}
I20260808 03:51:22.824095 31117 heartbeater.cc:347] Connected to a master server at 127.26.146.190:35137
I20260808 03:51:22.824368 31117 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:22.825060 31117 heartbeater.cc:511] Master 127.26.146.190:35137 requested a full tablet report, sending...
I20260808 03:51:22.827287 30834 ts_manager.cc:194] Registered new tserver with Master: 1c5b3a331d094fc380042a80e3bde1b3 (127.26.146.129:46455)
I20260808 03:51:22.829314 30834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.129@KRBTEST.COM'} at 127.26.146.129:48373
WARNING: no policy specified for kudu/127.26.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
I20260808 03:51:22.856411 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:35137
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:38829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:23.170493 31126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:23.170746 31126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:23.170868 31126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:23.177778 31126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:23.177874 31126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:23.177989 31126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:51:23.187079 31126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:38829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:35137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:23.188268 31126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:23.189519 31126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:23.201905 31143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:23.201946 31136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:23.202574 31126 server_base.cc:1034] running on GCE node
W20260808 03:51:23.202598 31135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:23.203372 31126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:23.204294 31126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:23.205490 31126 hybrid_clock.cc:648] HybridClock initialized: now 1786161083205441 us; error 40 us; skew 500 ppm
I20260808 03:51:23.205789 31126 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:23 dist-test-slave-jlzn krb5kdc[30767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161083, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:23.208743 31126 init.cc:377] Logged in from keytab as kudu/127.26.146.130@KRBTEST.COM (short username kudu)
I20260808 03:51:23.210098 31126 webserver.cc:460] Webserver started at http://127.26.146.130:38393/ using document root <none> and password file <none>
I20260808 03:51:23.210772 31126 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:23.210889 31126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:23.211186 31126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:23.214156 31126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "3252a2e5598d4e3aa3a1cbc2dc90caf9"
format_stamp: "Formatted at 2026-08-08 03:51:23 on dist-test-slave-jlzn"
server_key: "64d76baad7a33809811c66a577cd2139"
server_key_iv: "03bf01aa3eb57194e801155ef8862002"
server_key_version: "encryptionkey@0"
I20260808 03:51:23.214902 31126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "3252a2e5598d4e3aa3a1cbc2dc90caf9"
format_stamp: "Formatted at 2026-08-08 03:51:23 on dist-test-slave-jlzn"
server_key: "64d76baad7a33809811c66a577cd2139"
server_key_iv: "03bf01aa3eb57194e801155ef8862002"
server_key_version: "encryptionkey@0"
I20260808 03:51:23.219882 31126 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 03:51:23.223085 31155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:23.224356 31126 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 03:51:23.224500 31126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "3252a2e5598d4e3aa3a1cbc2dc90caf9"
format_stamp: "Formatted at 2026-08-08 03:51:23 on dist-test-slave-jlzn"
server_key: "64d76baad7a33809811c66a577cd2139"
server_key_iv: "03bf01aa3eb57194e801155ef8862002"
server_key_version: "encryptionkey@0"
I20260808 03:51:23.224684 31126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:23.258833 31126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:23.262563 31126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:23.262826 31126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:23.263759 31126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:23.265103 31126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:23.265192 31126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:23.265308 31126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:23.265370 31126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:23.294381 31126 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:38145
I20260808 03:51:23.294435 31319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:38145 every 8 connection(s)
I20260808 03:51:23.295630 31126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:51:23.304968 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 31126
I20260808 03:51:23.305111 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:51:23.305552 27210 external_mini_cluster.cc:1428] Setting key 4efd4180fd891223ab364c8f5de70b13
Aug 08 03:51:23 dist-test-slave-jlzn krb5kdc[30767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161083, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:23.315970 31121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:23.298509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:50007 (local address 127.26.146.190:35137)
0808 03:51:23.298717 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:23.298723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:23.299843 (+ 1120us) server_negotiation.cc:407] Connection header received
0808 03:51:23.301761 (+ 1918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:23.301766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:23.301834 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:23.301933 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:23.303429 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:23.304244 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:23.305823 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:23.306310 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:23.306415 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:23.310141 (+ 3726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:23.310147 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:23.310151 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:23.310198 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:23.313206 (+ 3008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:23.313864 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:23.313869 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:23.313872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:23.314026 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:23.314415 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:23.314419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:23.314421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:23.314764 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:23.314901 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:23.315512 (+ 611us) server_negotiation.cc:299] Negotiation successful
0808 03:51:23.315695 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 03:51:23.317278 31324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:23.298989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:35137 (local address 127.26.146.130:50007)
0808 03:51:23.299669 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:23.299715 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:23.301502 (+ 1787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:23.302167 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:23.302175 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:23.302695 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:23.303245 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:23.303263 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:23.304372 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:23.304376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:23.305689 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:23.305702 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:23.306530 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:23.306539 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:23.306755 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:23.307414 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:23.307447 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:23.309921 (+ 2474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:23.313334 (+ 3413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:23.313340 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:23.313353 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:23.313754 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:23.314112 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:23.314115 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:23.314118 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:23.314312 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:23.314860 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:23.314866 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:23.315174 (+ 308us) client_negotiation.cc:769] Sending connection context
0808 03:51:23.315391 (+ 217us) client_negotiation.cc:240] Negotiation successful
0808 03:51:23.315921 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":267,"threads_started":1}
I20260808 03:51:23.318661 31320 heartbeater.cc:347] Connected to a master server at 127.26.146.190:35137
I20260808 03:51:23.318940 31320 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:23.319729 31320 heartbeater.cc:511] Master 127.26.146.190:35137 requested a full tablet report, sending...
I20260808 03:51:23.321187 30834 ts_manager.cc:194] Registered new tserver with Master: 3252a2e5598d4e3aa3a1cbc2dc90caf9 (127.26.146.130:38145)
I20260808 03:51:23.321970 30834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.130@KRBTEST.COM'} at 127.26.146.130:50007
WARNING: no policy specified for kudu/127.26.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
I20260808 03:51:23.352854 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:35137
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:38829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:23.665221 31328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:23.665480 31328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:23.665582 31328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:23.672778 31328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:23.672896 31328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:23.673013 31328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:51:23.682196 31328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:38829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:35137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:23.683303 31328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:23.684542 31328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:23.696713 31336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:23.696745 31334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:23.696543 31338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:23.697877 31328 server_base.cc:1034] running on GCE node
I20260808 03:51:23.698469 31328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:23.699429 31328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:23.700637 31328 hybrid_clock.cc:648] HybridClock initialized: now 1786161083700588 us; error 42 us; skew 500 ppm
I20260808 03:51:23.700954 31328 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:23 dist-test-slave-jlzn krb5kdc[30767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161083, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:23.703994 31328 init.cc:377] Logged in from keytab as kudu/127.26.146.131@KRBTEST.COM (short username kudu)
I20260808 03:51:23.705304 31328 webserver.cc:460] Webserver started at http://127.26.146.131:46215/ using document root <none> and password file <none>
I20260808 03:51:23.705919 31328 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:23.706015 31328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:23.706269 31328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:23.709275 31328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "f4217c0e681e4b46b6b9ccd8acf56279"
format_stamp: "Formatted at 2026-08-08 03:51:23 on dist-test-slave-jlzn"
server_key: "a9d7d6bcd16a7ceb5e13e54bd272a420"
server_key_iv: "ba238fd5cbec833562e0add6281c6883"
server_key_version: "encryptionkey@0"
I20260808 03:51:23.710001 31328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "f4217c0e681e4b46b6b9ccd8acf56279"
format_stamp: "Formatted at 2026-08-08 03:51:23 on dist-test-slave-jlzn"
server_key: "a9d7d6bcd16a7ceb5e13e54bd272a420"
server_key_iv: "ba238fd5cbec833562e0add6281c6883"
server_key_version: "encryptionkey@0"
I20260808 03:51:23.714810 31328 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 03:51:23.718078 31350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:23.719372 31328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 03:51:23.719506 31328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "f4217c0e681e4b46b6b9ccd8acf56279"
format_stamp: "Formatted at 2026-08-08 03:51:23 on dist-test-slave-jlzn"
server_key: "a9d7d6bcd16a7ceb5e13e54bd272a420"
server_key_iv: "ba238fd5cbec833562e0add6281c6883"
server_key_version: "encryptionkey@0"
I20260808 03:51:23.719686 31328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:23.741923 31328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:23.745635 31328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:23.745887 31328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:23.746766 31328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:23.748037 31328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:23.748113 31328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:23.748214 31328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:23.748257 31328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:23.778075 31328 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:35935
I20260808 03:51:23.778151 31509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:35935 every 8 connection(s)
I20260808 03:51:23.779392 31328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:51:23.780272 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 31328
I20260808 03:51:23.780432 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:51:23.780956 27210 external_mini_cluster.cc:1428] Setting key 83fdfc96fb4056c17439cf61f8588e0a
Aug 08 03:51:23 dist-test-slave-jlzn krb5kdc[30767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161083, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:23.798218 31121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:23.782102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:33307 (local address 127.26.146.190:35137)
0808 03:51:23.782289 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:23.782296 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:23.783552 (+ 1256us) server_negotiation.cc:407] Connection header received
0808 03:51:23.785361 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:23.785367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:23.785445 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:23.785525 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:23.786950 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:23.787704 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:23.789189 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:23.789600 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:23.789710 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:23.793020 (+ 3310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:23.793025 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:23.793028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:23.793058 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:23.795477 (+ 2419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:23.796341 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:23.796347 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:23.796351 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:23.796470 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:23.796814 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:23.796817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:23.796819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:23.797067 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:23.797173 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:23.797811 (+ 638us) server_negotiation.cc:299] Negotiation successful
0808 03:51:23.797965 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 03:51:23.799054 31514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:23.782649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:35137 (local address 127.26.146.131:33307)
0808 03:51:23.783412 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:23.783454 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:23.785117 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:23.785765 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:23.785772 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:23.786264 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:23.786790 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:23.786807 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:23.787868 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:23.787872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:23.789050 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:23.789062 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:23.789814 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:23.789822 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:23.790034 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:23.790619 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:23.790640 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:23.792830 (+ 2190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:23.795611 (+ 2781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:23.795617 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:23.795631 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:23.796213 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:23.796551 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:23.796554 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:23.796557 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:23.796732 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:23.797164 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:23.797169 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:23.797498 (+ 329us) client_negotiation.cc:769] Sending connection context
0808 03:51:23.797711 (+ 213us) client_negotiation.cc:240] Negotiation successful
0808 03:51:23.798058 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":350,"threads_started":1}
I20260808 03:51:23.800232 31510 heartbeater.cc:347] Connected to a master server at 127.26.146.190:35137
I20260808 03:51:23.800500 31510 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:23.801421 31510 heartbeater.cc:511] Master 127.26.146.190:35137 requested a full tablet report, sending...
I20260808 03:51:23.802798 30834 ts_manager.cc:194] Registered new tserver with Master: f4217c0e681e4b46b6b9ccd8acf56279 (127.26.146.131:35935)
I20260808 03:51:23.803596 30834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.131@KRBTEST.COM'} at 127.26.146.131:33307
I20260808 03:51:23.808249 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:51:23.832366 31117 heartbeater.cc:503] Master 127.26.146.190:35137 was elected leader, sending a full tablet report...
W20260808 03:51:23.843225 31121 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:51:23.840837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46220 (local address 127.26.146.190:35137)
0808 03:51:23.841008 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:23.841015 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:23.841417 (+ 402us) server_negotiation.cc:407] Connection header received
0808 03:51:23.841585 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:23.841589 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:23.841652 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:23.841747 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:23.842978 (+ 1231us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46220: BlockingRecv error: recv got EOF from 127.0.0.1:46220 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 03:51:23.846280 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 30926
W20260808 03:51:23.848907 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 30926: Not authorized: Client connection negotiation failed: client connection to 127.26.146.129:46455: server requires authentication, but client does not have Kerberos credentials available
I20260808 03:51:23.849115 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 30926
I20260808 03:51:23.870965 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 31126
W20260808 03:51:23.873625 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 31126: Not authorized: Client connection negotiation failed: client connection to 127.26.146.130:38145: server requires authentication, but client does not have Kerberos credentials available
I20260808 03:51:23.873785 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 31126
I20260808 03:51:23.904551 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 31328
W20260808 03:51:23.907407 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 31328: Not authorized: Client connection negotiation failed: client connection to 127.26.146.131:35935: server requires authentication, but client does not have Kerberos credentials available
I20260808 03:51:23.907554 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 31328
I20260808 03:51:23.927194 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 30783
I20260808 03:51:23.928462 30876 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:24.029646 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 30783
2026-08-08T03:51:24Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3581 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:51:24 dist-test-slave-jlzn krb5kdc[31539](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:51:24 dist-test-slave-jlzn krb5kdc[31539](info): set up 2 sockets
Aug 08 03:51:24 dist-test-slave-jlzn krb5kdc[31539](info): commencing operation
krb5kdc: starting...
W20260808 03:51:25.213222 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.009s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:51:25 dist-test-slave-jlzn krb5kdc[31539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:51:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:51:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:51:25.411849 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:43081
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:36659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:43081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:25.730422 31558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:25.730732 31558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:25.730849 31558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:25.738034 31558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:51:25.738137 31558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:25.738214 31558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:25.738283 31558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:51:25.738349 31558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:51:25.747998 31558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:36659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:43081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:43081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:25.749230 31558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:25.750506 31558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:25.761672 31565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:25.761806 31566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:25.761782 31568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:25.762948 31558 server_base.cc:1034] running on GCE node
I20260808 03:51:25.763674 31558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:25.764977 31558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:25.766168 31558 hybrid_clock.cc:648] HybridClock initialized: now 1786161085766113 us; error 50 us; skew 500 ppm
I20260808 03:51:25.766485 31558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:25 dist-test-slave-jlzn krb5kdc[31539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161085, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:25.773262 31558 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:51:25.774619 31558 webserver.cc:460] Webserver started at http://127.26.146.190:37631/ using document root <none> and password file <none>
I20260808 03:51:25.775251 31558 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:25.775367 31558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:25.775660 31558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:25.778555 31558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "7b4c6bdd7e8f4e70b1fc5a4243c09427"
format_stamp: "Formatted at 2026-08-08 03:51:25 on dist-test-slave-jlzn"
server_key: "98fdbe10ac5f0fab06ab4ec6f1b01a41"
server_key_iv: "ae826469313dc015b04dfe9cf95f6658"
server_key_version: "encryptionkey@0"
I20260808 03:51:25.779318 31558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "7b4c6bdd7e8f4e70b1fc5a4243c09427"
format_stamp: "Formatted at 2026-08-08 03:51:25 on dist-test-slave-jlzn"
server_key: "98fdbe10ac5f0fab06ab4ec6f1b01a41"
server_key_iv: "ae826469313dc015b04dfe9cf95f6658"
server_key_version: "encryptionkey@0"
I20260808 03:51:25.784132 31558 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.002s
I20260808 03:51:25.787720 31575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:25.789155 31558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:51:25.789315 31558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "7b4c6bdd7e8f4e70b1fc5a4243c09427"
format_stamp: "Formatted at 2026-08-08 03:51:25 on dist-test-slave-jlzn"
server_key: "98fdbe10ac5f0fab06ab4ec6f1b01a41"
server_key_iv: "ae826469313dc015b04dfe9cf95f6658"
server_key_version: "encryptionkey@0"
I20260808 03:51:25.789497 31558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:25.810544 31558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:25.814385 31558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:25.814651 31558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:25.832145 31558 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:43081
I20260808 03:51:25.832186 31646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:43081 every 8 connection(s)
I20260808 03:51:25.833444 31558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:51:25.837592 31648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:25.838425 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 31558
I20260808 03:51:25.838665 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:51:25.839478 27210 external_mini_cluster.cc:1428] Setting key b2d7943a867525812c8164ecdb9a306b
I20260808 03:51:25.849964 31648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b4c6bdd7e8f4e70b1fc5a4243c09427: Bootstrap starting.
Aug 08 03:51:25 dist-test-slave-jlzn krb5kdc[31539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:25.853199 31648 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b4c6bdd7e8f4e70b1fc5a4243c09427: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:25.854187 31648 log.cc:824] T 00000000000000000000000000000000 P 7b4c6bdd7e8f4e70b1fc5a4243c09427: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:25.856925 31648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b4c6bdd7e8f4e70b1fc5a4243c09427: No bootstrap required, opened a new log
I20260808 03:51:25.860831 31651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:25.841320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52604 (local address 127.26.146.190:43081)
0808 03:51:25.842119 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:25.842134 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:25.842175 (+ 41us) server_negotiation.cc:407] Connection header received
0808 03:51:25.844015 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:25.844036 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:25.844409 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:25.844757 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:25.846230 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:25.847531 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:25.848870 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:25.849525 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:25.849750 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:25.853809 (+ 4059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:25.853817 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:25.853831 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:25.853866 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:25.856799 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:25.857490 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:25.857496 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:25.857502 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:25.857674 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:25.858035 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:25.858039 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:25.858041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:25.858617 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:25.858914 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:25.859359 (+ 445us) server_negotiation.cc:299] Negotiation successful
0808 03:51:25.859790 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":345,"threads_started":1}
I20260808 03:51:25.864521 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:51:25.864728 31648 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b4c6bdd7e8f4e70b1fc5a4243c09427 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b4c6bdd7e8f4e70b1fc5a4243c09427" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 43081 } }
I20260808 03:51:25.865062 31648 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b4c6bdd7e8f4e70b1fc5a4243c09427 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:25.865149 31648 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b4c6bdd7e8f4e70b1fc5a4243c09427 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b4c6bdd7e8f4e70b1fc5a4243c09427, State: Initialized, Role: FOLLOWER
I20260808 03:51:25.865643 31648 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b4c6bdd7e8f4e70b1fc5a4243c09427 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b4c6bdd7e8f4e70b1fc5a4243c09427" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 43081 } }
I20260808 03:51:25.865856 31648 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b4c6bdd7e8f4e70b1fc5a4243c09427 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:51:25.865973 31648 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b4c6bdd7e8f4e70b1fc5a4243c09427 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:51:25.866114 31648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b4c6bdd7e8f4e70b1fc5a4243c09427 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:25.868284 31648 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b4c6bdd7e8f4e70b1fc5a4243c09427 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b4c6bdd7e8f4e70b1fc5a4243c09427" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 43081 } }
I20260808 03:51:25.868762 31648 leader_election.cc:302] T 00000000000000000000000000000000 P 7b4c6bdd7e8f4e70b1fc5a4243c09427 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b4c6bdd7e8f4e70b1fc5a4243c09427; no voters:
I20260808 03:51:25.869364 31648 leader_election.cc:288] T 00000000000000000000000000000000 P 7b4c6bdd7e8f4e70b1fc5a4243c09427 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:51:25.869468 31653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b4c6bdd7e8f4e70b1fc5a4243c09427 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:25.871585 31648 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b4c6bdd7e8f4e70b1fc5a4243c09427 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:51:25.876103 31653 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b4c6bdd7e8f4e70b1fc5a4243c09427 [term 1 LEADER]: Becoming Leader. State: Replica: 7b4c6bdd7e8f4e70b1fc5a4243c09427, State: Running, Role: LEADER
I20260808 03:51:25.876834 31653 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b4c6bdd7e8f4e70b1fc5a4243c09427 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b4c6bdd7e8f4e70b1fc5a4243c09427" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 43081 } }
I20260808 03:51:25.879689 31655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b4c6bdd7e8f4e70b1fc5a4243c09427 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b4c6bdd7e8f4e70b1fc5a4243c09427" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b4c6bdd7e8f4e70b1fc5a4243c09427" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 43081 } } }
I20260808 03:51:25.879700 31656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b4c6bdd7e8f4e70b1fc5a4243c09427 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b4c6bdd7e8f4e70b1fc5a4243c09427. Latest consensus state: current_term: 1 leader_uuid: "7b4c6bdd7e8f4e70b1fc5a4243c09427" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b4c6bdd7e8f4e70b1fc5a4243c09427" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 43081 } } }
I20260808 03:51:25.880172 31655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b4c6bdd7e8f4e70b1fc5a4243c09427 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:25.880245 31656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b4c6bdd7e8f4e70b1fc5a4243c09427 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:25.886965 31669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:51:25.891376 31669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:51:25.901263 31669 catalog_manager.cc:1347] Generated new cluster ID: 604b7be1bea54c158e64acf60fb86ac7
I20260808 03:51:25.901360 31669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:51:25.915313 31669 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:51:25.917011 31669 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:51:25.924551 31669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b4c6bdd7e8f4e70b1fc5a4243c09427: Generated new TSK 0
I20260808 03:51:25.925532 31669 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:51:25.926028 31651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:25.916129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52616 (local address 127.26.146.190:43081)
0808 03:51:25.916285 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:25.916291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:25.916792 (+ 501us) server_negotiation.cc:407] Connection header received
0808 03:51:25.916986 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:25.916996 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:25.917082 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:25.917193 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:25.918607 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:25.919756 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:25.921805 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:25.922441 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:25.922586 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:25.924281 (+ 1695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:25.924287 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:25.924292 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:51:25.925185 (+ 893us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:25.925267 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:51:25.925305 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:25.925408 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:25.925587 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 03:51:25.925696 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 03:51:25.934190 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 31558
I20260808 03:51:25.935257 31642 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:26.021724 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 31558
2026-08-08T03:51:26Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1986 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:51:26 dist-test-slave-jlzn krb5kdc[31692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:51:26 dist-test-slave-jlzn krb5kdc[31692](info): set up 2 sockets
Aug 08 03:51:26 dist-test-slave-jlzn krb5kdc[31692](info): commencing operation
krb5kdc: starting...
W20260808 03:51:27.197521 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:51:27 dist-test-slave-jlzn krb5kdc[31692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:51:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:51:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:51:27.399991 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:43655
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:43655
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:27.963232 31718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:27.963915 31718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:27.964133 31718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:27.976442 31718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:51:27.976605 31718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:27.976732 31718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:27.976877 31718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:51:27.976976 31718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:51:27.994158 31718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:43655
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:43655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:27.996567 31718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:27.998842 31718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:28.018440 31731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:28.018507 31734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:28.018566 31732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:28.020437 31718 server_base.cc:1034] running on GCE node
I20260808 03:51:28.021979 31718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:28.024873 31718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:28.026221 31718 hybrid_clock.cc:648] HybridClock initialized: now 1786161088026113 us; error 84 us; skew 500 ppm
I20260808 03:51:28.026973 31718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:28 dist-test-slave-jlzn krb5kdc[31692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161088, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:28.034090 31718 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:51:28.037417 31718 webserver.cc:460] Webserver started at http://127.26.146.190:42459/ using document root <none> and password file <none>
I20260808 03:51:28.038559 31718 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:28.038812 31718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:28.039366 31718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:28.045162 31718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "7a0823f42f9041ae8fa5be410339b2e1"
format_stamp: "Formatted at 2026-08-08 03:51:28 on dist-test-slave-jlzn"
server_key: "3e35595ffb95a4699d319d9c7ea41661"
server_key_iv: "ee44a0a2c21faa244bf694832bdc5449"
server_key_version: "encryptionkey@0"
I20260808 03:51:28.046453 31718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "7a0823f42f9041ae8fa5be410339b2e1"
format_stamp: "Formatted at 2026-08-08 03:51:28 on dist-test-slave-jlzn"
server_key: "3e35595ffb95a4699d319d9c7ea41661"
server_key_iv: "ee44a0a2c21faa244bf694832bdc5449"
server_key_version: "encryptionkey@0"
I20260808 03:51:28.056979 31718 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.010s sys 0.001s
I20260808 03:51:28.063192 31744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:28.065747 31718 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260808 03:51:28.065973 31718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "7a0823f42f9041ae8fa5be410339b2e1"
format_stamp: "Formatted at 2026-08-08 03:51:28 on dist-test-slave-jlzn"
server_key: "3e35595ffb95a4699d319d9c7ea41661"
server_key_iv: "ee44a0a2c21faa244bf694832bdc5449"
server_key_version: "encryptionkey@0"
I20260808 03:51:28.066303 31718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:28.093060 31718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:28.099066 31718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:28.099421 31718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:28.124235 31718 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:43655
I20260808 03:51:28.124357 31815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:43655 every 8 connection(s)
I20260808 03:51:28.126305 31718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:51:28.132223 31817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:28.135071 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 31718
I20260808 03:51:28.135444 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:51:28.136771 27210 external_mini_cluster.cc:1428] Setting key 141f7375d1bf8e43b71bb7b6548e3c4b
I20260808 03:51:28.153435 31817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a0823f42f9041ae8fa5be410339b2e1: Bootstrap starting.
Aug 08 03:51:28 dist-test-slave-jlzn krb5kdc[31692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:28.158854 31817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a0823f42f9041ae8fa5be410339b2e1: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:28.160341 31817 log.cc:824] T 00000000000000000000000000000000 P 7a0823f42f9041ae8fa5be410339b2e1: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:28.165149 31817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a0823f42f9041ae8fa5be410339b2e1: No bootstrap required, opened a new log
I20260808 03:51:28.170838 31820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:28.140343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41866 (local address 127.26.146.190:43655)
0808 03:51:28.141662 (+ 1319us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:28.141685 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:28.141773 (+ 88us) server_negotiation.cc:407] Connection header received
0808 03:51:28.144479 (+ 2706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:28.144509 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:28.145087 (+ 578us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:28.145644 (+ 557us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:28.147980 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:28.149999 (+ 2019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:28.151860 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:28.153008 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:28.153287 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:28.159757 (+ 6470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:28.159769 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:28.159792 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:28.159885 (+ 93us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:28.164640 (+ 4755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:28.165668 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:28.165683 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:28.165695 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:28.165964 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:28.166494 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:28.166500 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:28.166502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:28.167402 (+ 900us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:28.167895 (+ 493us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:28.168554 (+ 659us) server_negotiation.cc:299] Negotiation successful
0808 03:51:28.169277 (+ 723us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":969,"thread_start_us":550,"threads_started":1}
I20260808 03:51:28.177234 31817 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a0823f42f9041ae8fa5be410339b2e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a0823f42f9041ae8fa5be410339b2e1" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 43655 } }
I20260808 03:51:28.177799 31817 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a0823f42f9041ae8fa5be410339b2e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:28.177968 31817 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a0823f42f9041ae8fa5be410339b2e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0823f42f9041ae8fa5be410339b2e1, State: Initialized, Role: FOLLOWER
I20260808 03:51:28.178872 31817 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a0823f42f9041ae8fa5be410339b2e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a0823f42f9041ae8fa5be410339b2e1" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 43655 } }
I20260808 03:51:28.179188 31817 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a0823f42f9041ae8fa5be410339b2e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:51:28.179358 31817 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a0823f42f9041ae8fa5be410339b2e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:51:28.179621 31817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a0823f42f9041ae8fa5be410339b2e1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:28.183525 31817 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a0823f42f9041ae8fa5be410339b2e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a0823f42f9041ae8fa5be410339b2e1" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 43655 } }
I20260808 03:51:28.184430 31817 leader_election.cc:302] T 00000000000000000000000000000000 P 7a0823f42f9041ae8fa5be410339b2e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a0823f42f9041ae8fa5be410339b2e1; no voters:
I20260808 03:51:28.185622 31817 leader_election.cc:288] T 00000000000000000000000000000000 P 7a0823f42f9041ae8fa5be410339b2e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:51:28.185814 31825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a0823f42f9041ae8fa5be410339b2e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:28.187723 31825 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a0823f42f9041ae8fa5be410339b2e1 [term 1 LEADER]: Becoming Leader. State: Replica: 7a0823f42f9041ae8fa5be410339b2e1, State: Running, Role: LEADER
I20260808 03:51:28.188625 31825 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a0823f42f9041ae8fa5be410339b2e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a0823f42f9041ae8fa5be410339b2e1" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 43655 } }
I20260808 03:51:28.189596 31817 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a0823f42f9041ae8fa5be410339b2e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:51:28.193514 31826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a0823f42f9041ae8fa5be410339b2e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a0823f42f9041ae8fa5be410339b2e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a0823f42f9041ae8fa5be410339b2e1" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 43655 } } }
I20260808 03:51:28.194528 31826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a0823f42f9041ae8fa5be410339b2e1 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:28.193817 31828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a0823f42f9041ae8fa5be410339b2e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a0823f42f9041ae8fa5be410339b2e1. Latest consensus state: current_term: 1 leader_uuid: "7a0823f42f9041ae8fa5be410339b2e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a0823f42f9041ae8fa5be410339b2e1" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 43655 } } }
I20260808 03:51:28.195345 31828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a0823f42f9041ae8fa5be410339b2e1 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:28.200287 31840 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:51:28.210778 31840 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:51:28.238678 31840 catalog_manager.cc:1347] Generated new cluster ID: 05da98179da946a79fb262dfb9bf7d53
I20260808 03:51:28.239041 31840 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:51:28.270468 31840 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:51:28.277124 31840 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:51:28.318550 31840 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a0823f42f9041ae8fa5be410339b2e1: Generated new TSK 0
I20260808 03:51:28.322010 31840 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
W20260808 03:51:28.519260 27210 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.26.146.129: real 0.177s user 0.009s sys 0.052s
I20260808 03:51:28.525832 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:43655
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:29.037619 31860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:29.037887 31860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:29.037998 31860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:29.044951 31860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:29.045053 31860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:29.045168 31860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:51:29.054396 31860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:43655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:29.055526 31860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:29.056787 31860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:29.068908 31866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:29.068956 31871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:29.068985 31868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:29.069741 31860 server_base.cc:1034] running on GCE node
I20260808 03:51:29.070268 31860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:29.071085 31860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:29.072280 31860 hybrid_clock.cc:648] HybridClock initialized: now 1786161089072230 us; error 39 us; skew 500 ppm
I20260808 03:51:29.072587 31860 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:29 dist-test-slave-jlzn krb5kdc[31692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161089, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:29.075629 31860 init.cc:377] Logged in from keytab as kudu/127.26.146.129@KRBTEST.COM (short username kudu)
I20260808 03:51:29.077015 31860 webserver.cc:460] Webserver started at http://127.26.146.129:37201/ using document root <none> and password file <none>
I20260808 03:51:29.077657 31860 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:29.077775 31860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:29.078053 31860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:29.080860 31860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "2809a414b9e2461abdd068f35d6e350a"
format_stamp: "Formatted at 2026-08-08 03:51:29 on dist-test-slave-jlzn"
server_key: "66bbd1a65567e133487866bcd3a03692"
server_key_iv: "fa95a8b4e438daae30cd854d00f7aaf0"
server_key_version: "encryptionkey@0"
I20260808 03:51:29.081601 31860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "2809a414b9e2461abdd068f35d6e350a"
format_stamp: "Formatted at 2026-08-08 03:51:29 on dist-test-slave-jlzn"
server_key: "66bbd1a65567e133487866bcd3a03692"
server_key_iv: "fa95a8b4e438daae30cd854d00f7aaf0"
server_key_version: "encryptionkey@0"
I20260808 03:51:29.086314 31860 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 03:51:29.089533 31885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:29.090770 31860 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 03:51:29.090919 31860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "2809a414b9e2461abdd068f35d6e350a"
format_stamp: "Formatted at 2026-08-08 03:51:29 on dist-test-slave-jlzn"
server_key: "66bbd1a65567e133487866bcd3a03692"
server_key_iv: "fa95a8b4e438daae30cd854d00f7aaf0"
server_key_version: "encryptionkey@0"
I20260808 03:51:29.091140 31860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:29.109416 31860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:29.113158 31860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:29.113416 31860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:29.114286 31860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:29.115548 31860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:29.115629 31860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:29.115724 31860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:29.115767 31860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:29.146696 31860 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:40575
I20260808 03:51:29.146719 32044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:40575 every 8 connection(s)
I20260808 03:51:29.147961 31860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:51:29.148409 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 31860
I20260808 03:51:29.148582 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:51:29.149430 27210 external_mini_cluster.cc:1428] Setting key 4c91fb8c7f4dcb1962524c96f98a1cb8
Aug 08 03:51:29 dist-test-slave-jlzn krb5kdc[31692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161089, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:29.167392 32050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:29.150764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:53113 (local address 127.26.146.190:43655)
0808 03:51:29.153984 (+ 3220us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:29.153988 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:29.154003 (+ 15us) server_negotiation.cc:407] Connection header received
0808 03:51:29.154047 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:29.154051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:29.154117 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:29.154209 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:29.155679 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:29.156497 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:29.157982 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:29.158420 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:29.158527 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:29.161961 (+ 3434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:29.161966 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:29.161970 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:29.162001 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:29.164546 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:29.165202 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:29.165207 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:29.165211 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:29.165332 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:29.165734 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:29.165739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:29.165742 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:29.166005 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:29.166146 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:29.166769 (+ 623us) server_negotiation.cc:299] Negotiation successful
0808 03:51:29.166984 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3111,"thread_start_us":468,"threads_started":1}
I20260808 03:51:29.168315 32051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:29.150883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:43655 (local address 127.26.146.129:53113)
0808 03:51:29.151916 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:29.151970 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:29.153573 (+ 1603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:29.154480 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:29.154488 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:29.154969 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:29.155502 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:29.155520 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:29.156651 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:29.156655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:29.157848 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:29.157860 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:29.158654 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:29.158661 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:29.158881 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:29.159503 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:29.159528 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:29.161737 (+ 2209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:29.164685 (+ 2948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:29.164694 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:29.164709 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:29.165091 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:29.165434 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:29.165439 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:29.165442 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:29.165631 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:29.166125 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:29.166132 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:29.166438 (+ 306us) client_negotiation.cc:769] Sending connection context
0808 03:51:29.166677 (+ 239us) client_negotiation.cc:240] Negotiation successful
0808 03:51:29.167159 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":472,"threads_started":1}
WARNING: no policy specified for kudu/127.26.146.130@KRBTEST.COM; defaulting to no policy
I20260808 03:51:29.169844 32045 heartbeater.cc:347] Connected to a master server at 127.26.146.190:43655
I20260808 03:51:29.170117 32045 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:29.170883 32045 heartbeater.cc:511] Master 127.26.146.190:43655 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.130@KRBTEST.COM" created.
I20260808 03:51:29.173053 31768 ts_manager.cc:194] Registered new tserver with Master: 2809a414b9e2461abdd068f35d6e350a (127.26.146.129:40575)
I20260808 03:51:29.174669 31768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.129@KRBTEST.COM'} at 127.26.146.129:53113
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
I20260808 03:51:29.194991 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:43655
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:29.508026 32058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:29.508285 32058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:29.508417 32058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:29.515345 32058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:29.515441 32058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:29.515584 32058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:51:29.524730 32058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:43655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:29.525928 32058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:29.527179 32058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:29.539459 32066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:29.539449 32068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:29.539512 32074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:29.540329 32058 server_base.cc:1034] running on GCE node
I20260808 03:51:29.540858 32058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:29.541651 32058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:29.542846 32058 hybrid_clock.cc:648] HybridClock initialized: now 1786161089542807 us; error 29 us; skew 500 ppm
I20260808 03:51:29.543129 32058 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:29 dist-test-slave-jlzn krb5kdc[31692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161089, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:29.546072 32058 init.cc:377] Logged in from keytab as kudu/127.26.146.130@KRBTEST.COM (short username kudu)
I20260808 03:51:29.547407 32058 webserver.cc:460] Webserver started at http://127.26.146.130:34651/ using document root <none> and password file <none>
I20260808 03:51:29.548051 32058 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:29.548153 32058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:29.548400 32058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:29.551088 32058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "63cc102d683d47d9b21fb0123b289555"
format_stamp: "Formatted at 2026-08-08 03:51:29 on dist-test-slave-jlzn"
server_key: "746841cca8cb7eb524ddaf6eca7a4b4c"
server_key_iv: "1f2e542aadae704c2a76fed637eba87d"
server_key_version: "encryptionkey@0"
I20260808 03:51:29.551846 32058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "63cc102d683d47d9b21fb0123b289555"
format_stamp: "Formatted at 2026-08-08 03:51:29 on dist-test-slave-jlzn"
server_key: "746841cca8cb7eb524ddaf6eca7a4b4c"
server_key_iv: "1f2e542aadae704c2a76fed637eba87d"
server_key_version: "encryptionkey@0"
I20260808 03:51:29.556651 32058 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 03:51:29.559784 32086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:29.561136 32058 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 03:51:29.561268 32058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "63cc102d683d47d9b21fb0123b289555"
format_stamp: "Formatted at 2026-08-08 03:51:29 on dist-test-slave-jlzn"
server_key: "746841cca8cb7eb524ddaf6eca7a4b4c"
server_key_iv: "1f2e542aadae704c2a76fed637eba87d"
server_key_version: "encryptionkey@0"
I20260808 03:51:29.561424 32058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:29.578207 32058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:29.581754 32058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:29.582005 32058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:29.582870 32058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:29.584180 32058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:29.584255 32058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:29.584367 32058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:29.584416 32058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:29.614156 32058 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:44921
I20260808 03:51:29.614212 32242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:44921 every 8 connection(s)
I20260808 03:51:29.615370 32058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:51:29.622658 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 32058
I20260808 03:51:29.622828 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:51:29.623286 27210 external_mini_cluster.cc:1428] Setting key 5e426be682e1549f0ef78544e0506166
Aug 08 03:51:29 dist-test-slave-jlzn krb5kdc[31692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161089, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:29.635178 32050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:29.617839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:53725 (local address 127.26.146.190:43655)
0808 03:51:29.618013 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:29.618017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:29.619073 (+ 1056us) server_negotiation.cc:407] Connection header received
0808 03:51:29.620865 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:29.620871 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:29.620950 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:29.621036 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:29.622577 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:29.623358 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:29.625349 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:29.625780 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:29.625874 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:29.629578 (+ 3704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:29.629597 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:29.629601 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:29.629633 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:29.632223 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:29.632916 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:29.632920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:29.632922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:29.633043 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:29.633453 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:29.633456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:29.633458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:29.633745 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:29.633876 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:29.634564 (+ 688us) server_negotiation.cc:299] Negotiation successful
0808 03:51:29.634793 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 03:51:29.636184 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:29.618246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:43655 (local address 127.26.146.130:53725)
0808 03:51:29.618925 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:29.618962 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:29.620601 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:29.621284 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:29.621292 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:29.621779 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:29.622384 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:29.622411 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:29.623521 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:29.623526 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:29.625211 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:29.625227 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:29.626009 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:29.626016 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:29.626263 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:29.626930 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:29.626965 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:29.629348 (+ 2383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:29.632360 (+ 3012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:29.632369 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:29.632385 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:29.632794 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:29.633146 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:29.633151 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:29.633155 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:29.633340 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:29.633868 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:29.633875 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:29.634195 (+ 320us) client_negotiation.cc:769] Sending connection context
0808 03:51:29.634438 (+ 243us) client_negotiation.cc:240] Negotiation successful
0808 03:51:29.634974 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":261,"threads_started":1}
I20260808 03:51:29.637559 32243 heartbeater.cc:347] Connected to a master server at 127.26.146.190:43655
I20260808 03:51:29.637837 32243 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:29.638535 32243 heartbeater.cc:511] Master 127.26.146.190:43655 requested a full tablet report, sending...
I20260808 03:51:29.639974 31768 ts_manager.cc:194] Registered new tserver with Master: 63cc102d683d47d9b21fb0123b289555 (127.26.146.130:44921)
I20260808 03:51:29.641086 31768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.130@KRBTEST.COM'} at 127.26.146.130:53725
WARNING: no policy specified for kudu/127.26.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
I20260808 03:51:29.678427 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:43655
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:29.997566 32249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:29.997824 32249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:29.997954 32249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:30.005016 32249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:30.005117 32249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:30.005263 32249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:51:30.014596 32249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:43655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:30.015794 32249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:30.017127 32249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:30.029239 32258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:30.029294 32259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:30.029322 32263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:30.030089 32249 server_base.cc:1034] running on GCE node
I20260808 03:51:30.030635 32249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:30.031436 32249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:30.032647 32249 hybrid_clock.cc:648] HybridClock initialized: now 1786161090032593 us; error 43 us; skew 500 ppm
I20260808 03:51:30.032938 32249 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:30 dist-test-slave-jlzn krb5kdc[31692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161090, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:30.035923 32249 init.cc:377] Logged in from keytab as kudu/127.26.146.131@KRBTEST.COM (short username kudu)
I20260808 03:51:30.037206 32249 webserver.cc:460] Webserver started at http://127.26.146.131:46297/ using document root <none> and password file <none>
I20260808 03:51:30.037817 32249 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:30.037932 32249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:30.038236 32249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:30.040989 32249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "28ea81b8b723408bb96df036fc06fc18"
format_stamp: "Formatted at 2026-08-08 03:51:30 on dist-test-slave-jlzn"
server_key: "a0df93be110d3e992c55de03d48093b7"
server_key_iv: "41ae3e7bd9a34db3c4b6bd27968572a2"
server_key_version: "encryptionkey@0"
I20260808 03:51:30.041729 32249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "28ea81b8b723408bb96df036fc06fc18"
format_stamp: "Formatted at 2026-08-08 03:51:30 on dist-test-slave-jlzn"
server_key: "a0df93be110d3e992c55de03d48093b7"
server_key_iv: "41ae3e7bd9a34db3c4b6bd27968572a2"
server_key_version: "encryptionkey@0"
I20260808 03:51:30.046619 32249 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260808 03:51:30.049844 32273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:30.051028 32249 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 03:51:30.051188 32249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "28ea81b8b723408bb96df036fc06fc18"
format_stamp: "Formatted at 2026-08-08 03:51:30 on dist-test-slave-jlzn"
server_key: "a0df93be110d3e992c55de03d48093b7"
server_key_iv: "41ae3e7bd9a34db3c4b6bd27968572a2"
server_key_version: "encryptionkey@0"
I20260808 03:51:30.051368 32249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:30.071645 32249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:30.075511 32249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:30.075830 32249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:30.076730 32249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:30.077961 32249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:30.078048 32249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:30.078174 32249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:30.078228 32249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:30.107601 32249 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:37247
I20260808 03:51:30.107657 32443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:37247 every 8 connection(s)
I20260808 03:51:30.108933 32249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:51:30.116488 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 32249
I20260808 03:51:30.116639 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:51:30.117139 27210 external_mini_cluster.cc:1428] Setting key 8af5b9943b2714b3067ff429feaab99d
Aug 08 03:51:30 dist-test-slave-jlzn krb5kdc[31692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161090, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:30.129120 32050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:30.111511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:54973 (local address 127.26.146.190:43655)
0808 03:51:30.111710 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:30.111715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:30.112840 (+ 1125us) server_negotiation.cc:407] Connection header received
0808 03:51:30.114641 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:30.114647 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:30.114734 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:30.114821 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:30.116363 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:30.117161 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:30.118911 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:30.119525 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:30.119652 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:30.122935 (+ 3283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:30.122940 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:30.122944 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:30.122976 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:30.126170 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:30.126811 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:30.126816 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:30.126820 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:30.126969 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:30.127330 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:30.127335 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:30.127338 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:30.127755 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:30.127948 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:30.128574 (+ 626us) server_negotiation.cc:299] Negotiation successful
0808 03:51:30.128782 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 03:51:30.129837 32446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:30.112002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:43655 (local address 127.26.146.131:54973)
0808 03:51:30.112687 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:30.112726 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:30.114380 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:30.115049 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:30.115056 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:30.115544 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:30.116165 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:30.116191 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:30.117338 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:30.117342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:30.118764 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:30.118778 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:30.119723 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:30.119730 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:30.120011 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:30.120619 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:30.120648 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:30.122733 (+ 2085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:30.126310 (+ 3577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:30.126316 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:30.126327 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:30.126713 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:30.127057 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:30.127059 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:30.127062 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:30.127243 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:30.127912 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:30.127918 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:30.128207 (+ 289us) client_negotiation.cc:769] Sending connection context
0808 03:51:30.128424 (+ 217us) client_negotiation.cc:240] Negotiation successful
0808 03:51:30.128787 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":267,"threads_started":1}
I20260808 03:51:30.131258 32444 heartbeater.cc:347] Connected to a master server at 127.26.146.190:43655
I20260808 03:51:30.131585 32444 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:30.132438 32444 heartbeater.cc:511] Master 127.26.146.190:43655 requested a full tablet report, sending...
I20260808 03:51:30.133756 31768 ts_manager.cc:194] Registered new tserver with Master: 28ea81b8b723408bb96df036fc06fc18 (127.26.146.131:37247)
I20260808 03:51:30.134537 31768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.131@KRBTEST.COM'} at 127.26.146.131:54973
I20260808 03:51:30.144064 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 03:51:30 dist-test-slave-jlzn krb5kdc[31692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161090, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
I20260808 03:51:30.177430 32045 heartbeater.cc:503] Master 127.26.146.190:43655 was elected leader, sending a full tablet report...
Aug 08 03:51:30 dist-test-slave-jlzn krb5kdc[31692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161090, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:30.193781 32050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:30.181187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41880 (local address 127.26.146.190:43655)
0808 03:51:30.181334 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:30.181338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:30.181817 (+ 479us) server_negotiation.cc:407] Connection header received
0808 03:51:30.181955 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:30.181959 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:30.182017 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:30.182098 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:30.183395 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:30.184169 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:30.185324 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:30.185731 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:30.185819 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:30.189307 (+ 3488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:30.189312 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:30.189315 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:30.189346 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:30.191780 (+ 2434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:30.192333 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:30.192337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:30.192339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:30.192467 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:30.192732 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:30.192734 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:30.192736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:30.193033 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:30.193245 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:30.193404 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 03:51:30.193556 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 03:51:30.194317 31768 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41880
I20260808 03:51:30.195792 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 31860
Aug 08 03:51:30 dist-test-slave-jlzn krb5kdc[31692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161090, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.26.146.129@KRBTEST.COM
I20260808 03:51:30.210137 32463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:30.196990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47636 (local address 127.26.146.129:40575)
0808 03:51:30.197576 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:30.197583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:30.197606 (+ 23us) server_negotiation.cc:407] Connection header received
0808 03:51:30.197657 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:30.197663 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:30.197861 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:30.197974 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:30.198987 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:30.200094 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:30.201292 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:30.201822 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:30.201906 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:30.205345 (+ 3439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:30.205356 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:30.205370 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:30.205400 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:30.207938 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:30.208521 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:30.208527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:30.208529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:30.208642 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:30.208903 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:30.208906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:30.208908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:30.209224 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:30.209440 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:30.209597 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 03:51:30.209785 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":300,"threads_started":1}
W20260808 03:51:30.211349 32037 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:47636
W20260808 03:51:30.212404 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 31860: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:51:30.212618 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 31860
I20260808 03:51:30.234875 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 32058
Aug 08 03:51:30 dist-test-slave-jlzn krb5kdc[31692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161090, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.26.146.130@KRBTEST.COM
I20260808 03:51:30.249861 32464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:30.235740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60874 (local address 127.26.146.130:44921)
0808 03:51:30.236323 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:30.236330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:30.236357 (+ 27us) server_negotiation.cc:407] Connection header received
0808 03:51:30.236427 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:30.236433 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:30.236641 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:30.236777 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:30.238028 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:30.239044 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:30.240293 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:30.240831 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:30.240923 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:30.244854 (+ 3931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:30.244866 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:30.244881 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:30.244935 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:30.247483 (+ 2548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:30.248106 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:30.248113 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:30.248115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:30.248256 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:30.248549 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:30.248552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:30.248554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:30.248943 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:30.249168 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:30.249370 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 03:51:30.249566 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":300,"threads_started":1}
W20260808 03:51:30.251044 32238 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:60874
W20260808 03:51:30.252187 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 32058: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:51:30.252321 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 32058
I20260808 03:51:30.274574 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 32249
Aug 08 03:51:30 dist-test-slave-jlzn krb5kdc[31692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161090, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.26.146.131@KRBTEST.COM
I20260808 03:51:30.289867 32465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:30.275452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39516 (local address 127.26.146.131:37247)
0808 03:51:30.276086 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:30.276094 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:30.276125 (+ 31us) server_negotiation.cc:407] Connection header received
0808 03:51:30.276186 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:30.276192 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:30.276402 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:30.276528 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:30.277808 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:30.278900 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:30.280144 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:30.280708 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:30.280802 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:30.284796 (+ 3994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:30.284807 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:30.284823 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:30.284858 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:30.287509 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:30.288122 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:30.288130 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:30.288132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:30.288250 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:30.288528 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:30.288531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:30.288533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:30.288869 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:30.289090 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:30.289314 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 03:51:30.289504 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":290,"threads_started":1}
W20260808 03:51:30.291199 32438 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:39516
W20260808 03:51:30.292292 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 32249: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:51:30.292447 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 32249
I20260808 03:51:30.311228 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 31718
I20260808 03:51:30.312450 31811 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:30.415454 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 31718
2026-08-08T03:51:30Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4398 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:51:30 dist-test-slave-jlzn krb5kdc[32471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:51:30 dist-test-slave-jlzn krb5kdc[32471](info): set up 2 sockets
Aug 08 03:51:30 dist-test-slave-jlzn krb5kdc[32471](info): commencing operation
krb5kdc: starting...
W20260808 03:51:31.584223 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:51:31 dist-test-slave-jlzn krb5kdc[32471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:51:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:51:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:51:31.785576 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:40365
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:39103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:40365
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:32.111173 32486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:32.111459 32486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:32.111611 32486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:32.118788 32486 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:51:32.118892 32486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:32.118954 32486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:32.119011 32486 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:51:32.119055 32486 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:51:32.128831 32486 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:39103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:40365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:40365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:32.130043 32486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:32.131289 32486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:32.142550 32494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:32.142596 32496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:32.142621 32498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:32.143599 32486 server_base.cc:1034] running on GCE node
I20260808 03:51:32.144340 32486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:32.145607 32486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:32.146813 32486 hybrid_clock.cc:648] HybridClock initialized: now 1786161092146764 us; error 40 us; skew 500 ppm
I20260808 03:51:32.147097 32486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:32 dist-test-slave-jlzn krb5kdc[32471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161092, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:32.152753 32486 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:51:32.154089 32486 webserver.cc:460] Webserver started at http://127.26.146.190:41889/ using document root <none> and password file <none>
I20260808 03:51:32.154731 32486 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:32.154842 32486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:32.155146 32486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:32.158087 32486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "497cb7c16ca04c25944231d99b9f63f0"
format_stamp: "Formatted at 2026-08-08 03:51:32 on dist-test-slave-jlzn"
server_key: "4e42b7f1fa57e038090f3b7c0d25b105"
server_key_iv: "1df01f7c70228caaa4a51a2391040245"
server_key_version: "encryptionkey@0"
I20260808 03:51:32.158926 32486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "497cb7c16ca04c25944231d99b9f63f0"
format_stamp: "Formatted at 2026-08-08 03:51:32 on dist-test-slave-jlzn"
server_key: "4e42b7f1fa57e038090f3b7c0d25b105"
server_key_iv: "1df01f7c70228caaa4a51a2391040245"
server_key_version: "encryptionkey@0"
I20260808 03:51:32.163856 32486 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260808 03:51:32.167065 32512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:32.168346 32486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:51:32.168493 32486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "497cb7c16ca04c25944231d99b9f63f0"
format_stamp: "Formatted at 2026-08-08 03:51:32 on dist-test-slave-jlzn"
server_key: "4e42b7f1fa57e038090f3b7c0d25b105"
server_key_iv: "1df01f7c70228caaa4a51a2391040245"
server_key_version: "encryptionkey@0"
I20260808 03:51:32.168658 32486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:32.184595 32486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:32.188324 32486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:32.188581 32486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:32.205322 32486 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:40365
I20260808 03:51:32.205339 32581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:40365 every 8 connection(s)
I20260808 03:51:32.206594 32486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:51:32.210083 32582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:32.213378 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 32486
I20260808 03:51:32.213593 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:51:32.214233 27210 external_mini_cluster.cc:1428] Setting key 64689ddbd07dca1223251156270f9b2f
I20260808 03:51:32.222236 32582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 497cb7c16ca04c25944231d99b9f63f0: Bootstrap starting.
I20260808 03:51:32.225787 32582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 497cb7c16ca04c25944231d99b9f63f0: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:32.227026 32582 log.cc:824] T 00000000000000000000000000000000 P 497cb7c16ca04c25944231d99b9f63f0: Log is configured to *not* fsync() on all Append() calls
Aug 08 03:51:32 dist-test-slave-jlzn krb5kdc[32471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:32.229805 32582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 497cb7c16ca04c25944231d99b9f63f0: No bootstrap required, opened a new log
I20260808 03:51:32.236323 32586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:32.216058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40416 (local address 127.26.146.190:40365)
0808 03:51:32.216774 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:32.216785 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:32.216825 (+ 40us) server_negotiation.cc:407] Connection header received
0808 03:51:32.218384 (+ 1559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:32.218405 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:32.218776 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:32.219078 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:32.220908 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:32.222168 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:32.223868 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:32.224531 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:32.224652 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:32.229263 (+ 4611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:32.229271 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:32.229284 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:32.229319 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:32.232376 (+ 3057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:32.233031 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:32.233038 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:32.233044 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:32.233228 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:32.233596 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:32.233600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:32.233602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:32.234160 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:32.234464 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:32.234880 (+ 416us) server_negotiation.cc:299] Negotiation successful
0808 03:51:32.235282 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":325,"threads_started":1}
I20260808 03:51:32.237490 32582 raft_consensus.cc:348] T 00000000000000000000000000000000 P 497cb7c16ca04c25944231d99b9f63f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497cb7c16ca04c25944231d99b9f63f0" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 40365 } }
I20260808 03:51:32.237933 32582 raft_consensus.cc:374] T 00000000000000000000000000000000 P 497cb7c16ca04c25944231d99b9f63f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:32.238040 32582 raft_consensus.cc:729] T 00000000000000000000000000000000 P 497cb7c16ca04c25944231d99b9f63f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 497cb7c16ca04c25944231d99b9f63f0, State: Initialized, Role: FOLLOWER
I20260808 03:51:32.238667 32582 consensus_queue.cc:260] T 00000000000000000000000000000000 P 497cb7c16ca04c25944231d99b9f63f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497cb7c16ca04c25944231d99b9f63f0" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 40365 } }
I20260808 03:51:32.238956 32582 raft_consensus.cc:388] T 00000000000000000000000000000000 P 497cb7c16ca04c25944231d99b9f63f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:51:32.239106 32582 raft_consensus.cc:482] T 00000000000000000000000000000000 P 497cb7c16ca04c25944231d99b9f63f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:51:32.239264 32582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 497cb7c16ca04c25944231d99b9f63f0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:32.241343 32582 raft_consensus.cc:504] T 00000000000000000000000000000000 P 497cb7c16ca04c25944231d99b9f63f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497cb7c16ca04c25944231d99b9f63f0" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 40365 } }
I20260808 03:51:32.241776 32582 leader_election.cc:302] T 00000000000000000000000000000000 P 497cb7c16ca04c25944231d99b9f63f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 497cb7c16ca04c25944231d99b9f63f0; no voters:
I20260808 03:51:32.242399 32582 leader_election.cc:288] T 00000000000000000000000000000000 P 497cb7c16ca04c25944231d99b9f63f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:51:32.242497 32591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 497cb7c16ca04c25944231d99b9f63f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:32.243125 32591 raft_consensus.cc:686] T 00000000000000000000000000000000 P 497cb7c16ca04c25944231d99b9f63f0 [term 1 LEADER]: Becoming Leader. State: Replica: 497cb7c16ca04c25944231d99b9f63f0, State: Running, Role: LEADER
I20260808 03:51:32.243713 32591 consensus_queue.cc:237] T 00000000000000000000000000000000 P 497cb7c16ca04c25944231d99b9f63f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497cb7c16ca04c25944231d99b9f63f0" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 40365 } }
I20260808 03:51:32.244509 32582 sys_catalog.cc:564] T 00000000000000000000000000000000 P 497cb7c16ca04c25944231d99b9f63f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:51:32.246310 32593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 497cb7c16ca04c25944231d99b9f63f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 497cb7c16ca04c25944231d99b9f63f0. Latest consensus state: current_term: 1 leader_uuid: "497cb7c16ca04c25944231d99b9f63f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497cb7c16ca04c25944231d99b9f63f0" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 40365 } } }
I20260808 03:51:32.246678 32593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 497cb7c16ca04c25944231d99b9f63f0 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:32.247225 32592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 497cb7c16ca04c25944231d99b9f63f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "497cb7c16ca04c25944231d99b9f63f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497cb7c16ca04c25944231d99b9f63f0" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 40365 } } }
I20260808 03:51:32.247583 32592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 497cb7c16ca04c25944231d99b9f63f0 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:32.249321 32598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:51:32.254592 32598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:51:32.263432 32598 catalog_manager.cc:1347] Generated new cluster ID: 5d9b6a771dc54c21bc639b8369172fa5
I20260808 03:51:32.263537 32598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:51:32.273882 32598 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:51:32.275472 32598 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:51:32.283784 32598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 497cb7c16ca04c25944231d99b9f63f0: Generated new TSK 0
I20260808 03:51:32.284715 32598 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
I20260808 03:51:32.351701 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:40365
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:39103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:32.677510 32625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:32.677774 32625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:32.677874 32625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:32.684844 32625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:32.684947 32625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:32.685068 32625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:51:32.694607 32625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:39103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:40365
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:32.695832 32625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:32.697079 32625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:32.709600 32636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:32.709576 32638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:32.709524 32640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:32.710286 32625 server_base.cc:1034] running on GCE node
I20260808 03:51:32.710835 32625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:32.711735 32625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:32.712967 32625 hybrid_clock.cc:648] HybridClock initialized: now 1786161092712920 us; error 38 us; skew 500 ppm
I20260808 03:51:32.713280 32625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:32 dist-test-slave-jlzn krb5kdc[32471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161092, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:32.716391 32625 init.cc:377] Logged in from keytab as kudu/127.26.146.129@KRBTEST.COM (short username kudu)
I20260808 03:51:32.717720 32625 webserver.cc:460] Webserver started at http://127.26.146.129:38077/ using document root <none> and password file <none>
I20260808 03:51:32.718335 32625 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:32.718434 32625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:32.718695 32625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:32.721493 32625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "8d1c7557bbb148cf9733c0c6fa86597e"
format_stamp: "Formatted at 2026-08-08 03:51:32 on dist-test-slave-jlzn"
server_key: "36e69e5b7bf5de7be39dc6e00b35b973"
server_key_iv: "25bcc8292ef693bdec2a11d58b05980b"
server_key_version: "encryptionkey@0"
I20260808 03:51:32.722195 32625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "8d1c7557bbb148cf9733c0c6fa86597e"
format_stamp: "Formatted at 2026-08-08 03:51:32 on dist-test-slave-jlzn"
server_key: "36e69e5b7bf5de7be39dc6e00b35b973"
server_key_iv: "25bcc8292ef693bdec2a11d58b05980b"
server_key_version: "encryptionkey@0"
I20260808 03:51:32.726929 32625 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260808 03:51:32.730170 32648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:32.731371 32625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:51:32.731544 32625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "8d1c7557bbb148cf9733c0c6fa86597e"
format_stamp: "Formatted at 2026-08-08 03:51:32 on dist-test-slave-jlzn"
server_key: "36e69e5b7bf5de7be39dc6e00b35b973"
server_key_iv: "25bcc8292ef693bdec2a11d58b05980b"
server_key_version: "encryptionkey@0"
I20260808 03:51:32.731742 32625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:32.751191 32625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:32.755407 32625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:32.755663 32625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:32.756556 32625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:32.757790 32625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:32.757884 32625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:32.758018 32625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:32.758078 32625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:32.785985 32625 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:39843
I20260808 03:51:32.786055 335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:39843 every 8 connection(s)
I20260808 03:51:32.787246 32625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:51:32.792354 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 32625
I20260808 03:51:32.792541 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:51:32.793186 27210 external_mini_cluster.cc:1428] Setting key 1cccb47151dff451c9b7ecca211f9359
Aug 08 03:51:32 dist-test-slave-jlzn krb5kdc[32471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161092, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:32.806860 338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:32.789661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:51559 (local address 127.26.146.190:40365)
0808 03:51:32.790062 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:32.790066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:32.790940 (+ 874us) server_negotiation.cc:407] Connection header received
0808 03:51:32.792792 (+ 1852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:32.792798 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:32.792884 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:32.792979 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:32.794651 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:32.795675 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:32.797208 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:32.797758 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:32.797886 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:32.801110 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:32.801123 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:32.801127 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:32.801166 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:32.803903 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:32.804714 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:32.804718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:32.804720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:32.804837 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:32.805294 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:32.805297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:32.805299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:32.805539 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:32.805651 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:32.806407 (+ 756us) server_negotiation.cc:299] Negotiation successful
0808 03:51:32.806572 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":208,"threads_started":1}
I20260808 03:51:32.808162 339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:32.790107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:40365 (local address 127.26.146.129:51559)
0808 03:51:32.790794 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:32.790830 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:32.792514 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:32.793217 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:32.793225 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:32.793785 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:32.794467 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:32.794493 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:32.795876 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:32.795880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:32.797060 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:32.797073 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:32.797952 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:32.797957 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:32.798146 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:32.798758 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:32.798783 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:32.800902 (+ 2119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:32.804032 (+ 3130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:32.804041 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:32.804056 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:32.804612 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:32.804932 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:32.804937 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:32.804952 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:32.805205 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:32.805654 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:32.805660 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:32.806041 (+ 381us) client_negotiation.cc:769] Sending connection context
0808 03:51:32.806326 (+ 285us) client_negotiation.cc:240] Negotiation successful
0808 03:51:32.806864 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":251,"threads_started":1}
I20260808 03:51:32.809679 336 heartbeater.cc:347] Connected to a master server at 127.26.146.190:40365
I20260808 03:51:32.809960 336 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:32.810662 336 heartbeater.cc:511] Master 127.26.146.190:40365 requested a full tablet report, sending...
I20260808 03:51:32.812786 32537 ts_manager.cc:194] Registered new tserver with Master: 8d1c7557bbb148cf9733c0c6fa86597e (127.26.146.129:39843)
I20260808 03:51:32.814224 32537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.129@KRBTEST.COM'} at 127.26.146.129:51559
WARNING: no policy specified for kudu/127.26.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
I20260808 03:51:32.844866 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:40365
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:39103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:33.168809 343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:33.169100 343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:33.169234 343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:33.176523 343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:33.176651 343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:33.176801 343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:51:33.186395 343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:39103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:40365
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:33.187726 343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:33.189052 343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:33.201462 352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:33.201434 356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:33.201392 358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:33.202628 343 server_base.cc:1034] running on GCE node
I20260808 03:51:33.203234 343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:33.204193 343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:33.205380 343 hybrid_clock.cc:648] HybridClock initialized: now 1786161093205343 us; error 37 us; skew 500 ppm
I20260808 03:51:33.205657 343 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:33 dist-test-slave-jlzn krb5kdc[32471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161093, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:33.208783 343 init.cc:377] Logged in from keytab as kudu/127.26.146.130@KRBTEST.COM (short username kudu)
I20260808 03:51:33.212911 343 webserver.cc:460] Webserver started at http://127.26.146.130:39813/ using document root <none> and password file <none>
I20260808 03:51:33.213649 343 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:33.213766 343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:33.214042 343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:33.216876 343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "4dcedb5c7df74feab13b7ff4604c31ee"
format_stamp: "Formatted at 2026-08-08 03:51:33 on dist-test-slave-jlzn"
server_key: "8da1e7f74af76f52e5378c6f1d9f2350"
server_key_iv: "caecbe3e727127c24745e2d5501a513d"
server_key_version: "encryptionkey@0"
I20260808 03:51:33.217617 343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "4dcedb5c7df74feab13b7ff4604c31ee"
format_stamp: "Formatted at 2026-08-08 03:51:33 on dist-test-slave-jlzn"
server_key: "8da1e7f74af76f52e5378c6f1d9f2350"
server_key_iv: "caecbe3e727127c24745e2d5501a513d"
server_key_version: "encryptionkey@0"
I20260808 03:51:33.222512 343 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:51:33.225776 365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:33.226930 343 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 03:51:33.227080 343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "4dcedb5c7df74feab13b7ff4604c31ee"
format_stamp: "Formatted at 2026-08-08 03:51:33 on dist-test-slave-jlzn"
server_key: "8da1e7f74af76f52e5378c6f1d9f2350"
server_key_iv: "caecbe3e727127c24745e2d5501a513d"
server_key_version: "encryptionkey@0"
I20260808 03:51:33.227272 343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:33.245451 343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:33.249126 343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:33.249392 343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:33.250308 343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:33.251663 343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:33.251744 343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:33.251886 343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:33.251940 343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:33.281323 343 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:46533
I20260808 03:51:33.281353 543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:46533 every 8 connection(s)
I20260808 03:51:33.282517 343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:51:33.284759 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 343
I20260808 03:51:33.284953 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:51:33.285486 27210 external_mini_cluster.cc:1428] Setting key a78bcddd60dd4578cf1da64537b5097a
Aug 08 03:51:33 dist-test-slave-jlzn krb5kdc[32471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161093, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:33.302217 338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:33.285076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:42679 (local address 127.26.146.190:40365)
0808 03:51:33.285241 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:33.285246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:33.286669 (+ 1423us) server_negotiation.cc:407] Connection header received
0808 03:51:33.288789 (+ 2120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:33.288794 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:33.288868 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:33.288970 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:33.290441 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:33.291386 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:33.292914 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:33.293445 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:33.293561 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:33.296882 (+ 3321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:33.296887 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:33.296890 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:33.296922 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:33.299325 (+ 2403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:33.300042 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:33.300046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:33.300048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:33.300171 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:33.300574 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:33.300577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:33.300579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:33.300852 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:33.300984 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:33.301654 (+ 670us) server_negotiation.cc:299] Negotiation successful
0808 03:51:33.301889 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 03:51:33.303176 550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:33.285703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:40365 (local address 127.26.146.130:42679)
0808 03:51:33.286521 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:33.286558 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:33.288518 (+ 1960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:33.289193 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:33.289201 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:33.289716 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:33.290272 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:33.290289 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:33.291516 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:33.291520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:33.292771 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:33.292784 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:33.293636 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:33.293642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:33.293837 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:33.294444 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:33.294471 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:33.296674 (+ 2203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:33.299450 (+ 2776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:33.299459 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:33.299474 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:33.299927 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:33.300275 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:33.300280 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:33.300284 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:33.300463 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:33.300973 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:33.300980 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:33.301305 (+ 325us) client_negotiation.cc:769] Sending connection context
0808 03:51:33.301530 (+ 225us) client_negotiation.cc:240] Negotiation successful
0808 03:51:33.302023 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":309,"threads_started":1}
I20260808 03:51:33.304530 544 heartbeater.cc:347] Connected to a master server at 127.26.146.190:40365
I20260808 03:51:33.304811 544 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:33.305495 544 heartbeater.cc:511] Master 127.26.146.190:40365 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.26.146.131@KRBTEST.COM; defaulting to no policy
I20260808 03:51:33.306890 32537 ts_manager.cc:194] Registered new tserver with Master: 4dcedb5c7df74feab13b7ff4604c31ee (127.26.146.130:46533)
I20260808 03:51:33.307708 32537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.130@KRBTEST.COM'} at 127.26.146.130:42679
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
I20260808 03:51:33.334654 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:40365
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:39103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:33.654222 555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:33.654496 555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:33.654603 555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:33.662700 555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:33.662820 555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:33.662947 555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:51:33.672360 555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:39103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:40365
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:33.673609 555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:33.674851 555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:33.687354 562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:33.687408 563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:33.687409 567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:33.688129 555 server_base.cc:1034] running on GCE node
I20260808 03:51:33.688750 555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:33.689594 555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:33.690794 555 hybrid_clock.cc:648] HybridClock initialized: now 1786161093690749 us; error 36 us; skew 500 ppm
I20260808 03:51:33.691078 555 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:33 dist-test-slave-jlzn krb5kdc[32471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161093, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:33.694078 555 init.cc:377] Logged in from keytab as kudu/127.26.146.131@KRBTEST.COM (short username kudu)
I20260808 03:51:33.695477 555 webserver.cc:460] Webserver started at http://127.26.146.131:33523/ using document root <none> and password file <none>
I20260808 03:51:33.696188 555 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:33.696306 555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:33.696591 555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:33.699378 555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "40191e586afb4746b518ade1760ba565"
format_stamp: "Formatted at 2026-08-08 03:51:33 on dist-test-slave-jlzn"
server_key: "2a4c725d79f1cefe5624815ea20653f8"
server_key_iv: "16bb61438a744d7e1df9ac3fecf0e9e2"
server_key_version: "encryptionkey@0"
I20260808 03:51:33.700178 555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "40191e586afb4746b518ade1760ba565"
format_stamp: "Formatted at 2026-08-08 03:51:33 on dist-test-slave-jlzn"
server_key: "2a4c725d79f1cefe5624815ea20653f8"
server_key_iv: "16bb61438a744d7e1df9ac3fecf0e9e2"
server_key_version: "encryptionkey@0"
I20260808 03:51:33.704989 555 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 03:51:33.708355 579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:33.709554 555 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:51:33.709714 555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "40191e586afb4746b518ade1760ba565"
format_stamp: "Formatted at 2026-08-08 03:51:33 on dist-test-slave-jlzn"
server_key: "2a4c725d79f1cefe5624815ea20653f8"
server_key_iv: "16bb61438a744d7e1df9ac3fecf0e9e2"
server_key_version: "encryptionkey@0"
I20260808 03:51:33.709892 555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:33.739315 555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:33.743520 555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:33.743785 555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:33.744801 555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:33.746194 555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:33.746272 555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:33.746393 555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:33.746447 555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:33.776444 555 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:43391
I20260808 03:51:33.776495 734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:43391 every 8 connection(s)
I20260808 03:51:33.777693 555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:51:33.784372 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 555
I20260808 03:51:33.784540 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:51:33.784988 27210 external_mini_cluster.cc:1428] Setting key 0066587753dbe4d47c0eab74882c79d2
Aug 08 03:51:33 dist-test-slave-jlzn krb5kdc[32471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161093, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:33.797336 338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:33.780124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:57795 (local address 127.26.146.190:40365)
0808 03:51:33.780303 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:33.780308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:33.781474 (+ 1166us) server_negotiation.cc:407] Connection header received
0808 03:51:33.783305 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:33.783311 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:33.783387 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:33.783476 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:33.785488 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:33.786662 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:33.788458 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:33.789077 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:33.789205 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:33.792434 (+ 3229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:33.792439 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:33.792443 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:33.792475 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:33.794737 (+ 2262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:33.795368 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:33.795371 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:33.795373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:33.795491 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:33.795872 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:33.795875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:33.795877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:33.796159 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:33.796272 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:33.796867 (+ 595us) server_negotiation.cc:299] Negotiation successful
0808 03:51:33.797087 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 03:51:33.798110 738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:33.780615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:40365 (local address 127.26.146.131:57795)
0808 03:51:33.781307 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:33.781352 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:33.783024 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:33.783872 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:33.783881 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:33.784426 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:33.785238 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:33.785266 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:33.786826 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:33.786832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:33.788318 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:33.788330 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:33.789292 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:33.789297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:33.789485 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:33.790105 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:33.790127 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:33.792233 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:33.794866 (+ 2633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:33.794872 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:33.794882 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:33.795278 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:33.795565 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:33.795568 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:33.795570 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:33.795749 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:33.796257 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:33.796262 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:33.796538 (+ 276us) client_negotiation.cc:769] Sending connection context
0808 03:51:33.796756 (+ 218us) client_negotiation.cc:240] Negotiation successful
0808 03:51:33.797124 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":271,"threads_started":1}
I20260808 03:51:33.799557 736 heartbeater.cc:347] Connected to a master server at 127.26.146.190:40365
I20260808 03:51:33.799906 736 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:33.800678 736 heartbeater.cc:511] Master 127.26.146.190:40365 requested a full tablet report, sending...
I20260808 03:51:33.801999 32535 ts_manager.cc:194] Registered new tserver with Master: 40191e586afb4746b518ade1760ba565 (127.26.146.131:43391)
I20260808 03:51:33.802838 32535 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.131@KRBTEST.COM'} at 127.26.146.131:57795
I20260808 03:51:33.812314 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:51:33.817171 336 heartbeater.cc:503] Master 127.26.146.190:40365 was elected leader, sending a full tablet report...
Aug 08 03:51:33 dist-test-slave-jlzn krb5kdc[32471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 03:51:33 dist-test-slave-jlzn krb5kdc[32471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.129@KRBTEST.COM
I20260808 03:51:33.863780 743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:33.849119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58688 (local address 127.26.146.129:39843)
0808 03:51:33.849725 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:33.849732 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:33.849760 (+ 28us) server_negotiation.cc:407] Connection header received
0808 03:51:33.849822 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:33.849829 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:33.850032 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:33.850170 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:33.851633 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:33.852667 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:33.853772 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:33.854422 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:33.854518 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:33.858304 (+ 3786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:33.858315 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:33.858329 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:33.858365 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:33.861045 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:33.861723 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:33.861730 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:33.861732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:33.861857 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:33.862218 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:33.862221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:33.862223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:33.862578 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:33.862884 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:33.863163 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 03:51:33.863440 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":296,"threads_started":1}
Aug 08 03:51:33 dist-test-slave-jlzn krb5kdc[32471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:33.880569 338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:33.868187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40422 (local address 127.26.146.190:40365)
0808 03:51:33.868414 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:33.868420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:33.868570 (+ 150us) server_negotiation.cc:407] Connection header received
0808 03:51:33.868704 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:33.868708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:33.868771 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:33.868849 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:33.870012 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:33.870739 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:33.871964 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:33.872425 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:33.872522 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:33.875947 (+ 3425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:33.875952 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:33.875956 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:33.875988 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:33.878385 (+ 2397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:33.878936 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:33.878939 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:33.878941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:33.879071 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:33.879363 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:33.879366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:33.879368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:33.879662 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:33.879893 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:33.880120 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 03:51:33.880278 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260808 03:51:33.881083 32535 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40422
I20260808 03:51:33.882042 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 32625
I20260808 03:51:33.893374 743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:33.882891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58696 (local address 127.26.146.129:39843)
0808 03:51:33.883086 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:33.883092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:33.883224 (+ 132us) server_negotiation.cc:407] Connection header received
0808 03:51:33.883368 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:33.883371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:33.883461 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:33.883534 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:33.884630 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:33.885296 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:33.886394 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:33.886799 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:33.886888 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:33.888923 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:33.888928 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:33.888931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:33.888959 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:33.891264 (+ 2305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:33.891862 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:33.891865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:33.891868 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:33.891987 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:33.892289 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:33.892292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:33.892294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:33.892525 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:33.892695 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:33.892912 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 03:51:33.893068 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 03:51:33.893926 330 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:33.974570 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 32625
I20260808 03:51:33.997207 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 343
Aug 08 03:51:34 dist-test-slave-jlzn krb5kdc[32471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.130@KRBTEST.COM
I20260808 03:51:34.012734 751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:33.998111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54534 (local address 127.26.146.130:46533)
0808 03:51:33.998649 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:33.998657 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:33.998684 (+ 27us) server_negotiation.cc:407] Connection header received
0808 03:51:33.998770 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:33.998777 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:33.998978 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:33.999104 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:34.000293 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:34.001288 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:34.002454 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:34.002989 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:34.003081 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:34.007015 (+ 3934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:34.007026 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:34.007041 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:34.007078 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:34.009758 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:34.010309 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:34.010315 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:34.010317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:34.010436 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:34.010727 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:34.010730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:34.010732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:34.011045 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:34.011343 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:34.012219 (+ 876us) server_negotiation.cc:299] Negotiation successful
0808 03:51:34.012443 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":257,"threads_started":1}
I20260808 03:51:34.013949 535 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:34.093677 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 343
I20260808 03:51:34.116899 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 555
Aug 08 03:51:34 dist-test-slave-jlzn krb5kdc[32471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.131@KRBTEST.COM
I20260808 03:51:34.132404 753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:34.117836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47388 (local address 127.26.146.131:43391)
0808 03:51:34.118453 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:34.118460 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:34.118489 (+ 29us) server_negotiation.cc:407] Connection header received
0808 03:51:34.118549 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:34.118555 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:34.118780 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:34.118900 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:34.120254 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:34.121268 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:34.122462 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:34.123010 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:34.123104 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:34.127143 (+ 4039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:34.127155 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:34.127169 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:34.127215 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:34.129934 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:34.130493 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:34.130499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:34.130501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:34.130619 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:34.130933 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:34.130936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:34.130938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:34.131277 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:34.131528 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:34.131726 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 03:51:34.132080 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":291,"threads_started":1}
I20260808 03:51:34.133636 730 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:34.214530 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 555
I20260808 03:51:34.237357 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 32486
I20260808 03:51:34.238540 32575 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:34.340276 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 32486
2026-08-08T03:51:34Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3924 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:51:34 dist-test-slave-jlzn krb5kdc[761](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:51:34 dist-test-slave-jlzn krb5kdc[761](info): set up 2 sockets
Aug 08 03:51:34 dist-test-slave-jlzn krb5kdc[761](info): commencing operation
krb5kdc: starting...
W20260808 03:51:35.514843 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.007s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:51:35 dist-test-slave-jlzn krb5kdc[761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:51:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:51:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:51:35.725005 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:36557
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:40975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:36557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:36.043949 780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:36.044234 780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:36.044343 780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:36.051479 780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:51:36.051613 780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:36.051667 780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:36.051724 780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:51:36.051771 780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:51:36.061415 780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:40975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:36557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:36557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:36.062621 780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:36.063874 780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:36.074709 788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:36.074951 785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:36.075263 790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:36.075790 780 server_base.cc:1034] running on GCE node
I20260808 03:51:36.076583 780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:36.077857 780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:36.079064 780 hybrid_clock.cc:648] HybridClock initialized: now 1786161096079020 us; error 33 us; skew 500 ppm
I20260808 03:51:36.079429 780 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:36 dist-test-slave-jlzn krb5kdc[761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161096, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:36.082464 780 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:51:36.083575 780 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:36.083678 780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:36.084019 780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:36.086908 780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "bde73cb38c8545419aa9d2784e6f4621"
format_stamp: "Formatted at 2026-08-08 03:51:36 on dist-test-slave-jlzn"
server_key: "a3fbdab7402ccd0239c2a13bba4d5038"
server_key_iv: "39ec47fb5f9917af299a3382a5d643b4"
server_key_version: "encryptionkey@0"
I20260808 03:51:36.087692 780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "bde73cb38c8545419aa9d2784e6f4621"
format_stamp: "Formatted at 2026-08-08 03:51:36 on dist-test-slave-jlzn"
server_key: "a3fbdab7402ccd0239c2a13bba4d5038"
server_key_iv: "39ec47fb5f9917af299a3382a5d643b4"
server_key_version: "encryptionkey@0"
I20260808 03:51:36.092530 780 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 03:51:36.095985 801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:36.097187 780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 03:51:36.097362 780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "bde73cb38c8545419aa9d2784e6f4621"
format_stamp: "Formatted at 2026-08-08 03:51:36 on dist-test-slave-jlzn"
server_key: "a3fbdab7402ccd0239c2a13bba4d5038"
server_key_iv: "39ec47fb5f9917af299a3382a5d643b4"
server_key_version: "encryptionkey@0"
I20260808 03:51:36.097568 780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:36.118314 780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:36.122116 780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:36.122366 780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:36.139098 780 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:36557
I20260808 03:51:36.139129 869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:36557 every 8 connection(s)
I20260808 03:51:36.139966 780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:51:36.142293 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 780
I20260808 03:51:36.142494 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:51:36.143092 27210 external_mini_cluster.cc:1428] Setting key 89d1f09d6a06e72813e88b1190677a12
I20260808 03:51:36.143604 870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:36.157424 870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bde73cb38c8545419aa9d2784e6f4621: Bootstrap starting.
Aug 08 03:51:36 dist-test-slave-jlzn krb5kdc[761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:36.161254 870 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bde73cb38c8545419aa9d2784e6f4621: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:36.162575 870 log.cc:824] T 00000000000000000000000000000000 P bde73cb38c8545419aa9d2784e6f4621: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:36.165853 870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bde73cb38c8545419aa9d2784e6f4621: No bootstrap required, opened a new log
I20260808 03:51:36.167047 874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:36.145006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51226 (local address 127.26.146.190:36557)
0808 03:51:36.145845 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:36.145860 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:36.145908 (+ 48us) server_negotiation.cc:407] Connection header received
0808 03:51:36.148284 (+ 2376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:36.148318 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:36.148797 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:36.149231 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:36.150512 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:36.152533 (+ 2021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:36.153751 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:36.154819 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:36.155003 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:36.158867 (+ 3864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:36.158878 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:36.158900 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:36.158962 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:36.162756 (+ 3794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:36.163459 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:36.163466 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:36.163471 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:36.163640 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:36.164069 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:36.164074 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:36.164077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:36.164781 (+ 704us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:36.165067 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:36.165365 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 03:51:36.165768 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":365,"threads_started":1}
I20260808 03:51:36.173508 870 raft_consensus.cc:348] T 00000000000000000000000000000000 P bde73cb38c8545419aa9d2784e6f4621 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bde73cb38c8545419aa9d2784e6f4621" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36557 } }
I20260808 03:51:36.173840 870 raft_consensus.cc:374] T 00000000000000000000000000000000 P bde73cb38c8545419aa9d2784e6f4621 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:36.173920 870 raft_consensus.cc:729] T 00000000000000000000000000000000 P bde73cb38c8545419aa9d2784e6f4621 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bde73cb38c8545419aa9d2784e6f4621, State: Initialized, Role: FOLLOWER
I20260808 03:51:36.174422 870 consensus_queue.cc:260] T 00000000000000000000000000000000 P bde73cb38c8545419aa9d2784e6f4621 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bde73cb38c8545419aa9d2784e6f4621" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36557 } }
I20260808 03:51:36.174649 870 raft_consensus.cc:388] T 00000000000000000000000000000000 P bde73cb38c8545419aa9d2784e6f4621 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:51:36.174747 870 raft_consensus.cc:482] T 00000000000000000000000000000000 P bde73cb38c8545419aa9d2784e6f4621 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:51:36.174867 870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bde73cb38c8545419aa9d2784e6f4621 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:36.176963 870 raft_consensus.cc:504] T 00000000000000000000000000000000 P bde73cb38c8545419aa9d2784e6f4621 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bde73cb38c8545419aa9d2784e6f4621" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36557 } }
I20260808 03:51:36.177390 870 leader_election.cc:302] T 00000000000000000000000000000000 P bde73cb38c8545419aa9d2784e6f4621 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bde73cb38c8545419aa9d2784e6f4621; no voters:
I20260808 03:51:36.177906 870 leader_election.cc:288] T 00000000000000000000000000000000 P bde73cb38c8545419aa9d2784e6f4621 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:51:36.178045 876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bde73cb38c8545419aa9d2784e6f4621 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:36.178656 876 raft_consensus.cc:686] T 00000000000000000000000000000000 P bde73cb38c8545419aa9d2784e6f4621 [term 1 LEADER]: Becoming Leader. State: Replica: bde73cb38c8545419aa9d2784e6f4621, State: Running, Role: LEADER
I20260808 03:51:36.179193 876 consensus_queue.cc:237] T 00000000000000000000000000000000 P bde73cb38c8545419aa9d2784e6f4621 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bde73cb38c8545419aa9d2784e6f4621" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36557 } }
I20260808 03:51:36.179930 870 sys_catalog.cc:564] T 00000000000000000000000000000000 P bde73cb38c8545419aa9d2784e6f4621 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:51:36.181864 878 sys_catalog.cc:455] T 00000000000000000000000000000000 P bde73cb38c8545419aa9d2784e6f4621 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bde73cb38c8545419aa9d2784e6f4621" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bde73cb38c8545419aa9d2784e6f4621" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36557 } } }
I20260808 03:51:36.182305 878 sys_catalog.cc:458] T 00000000000000000000000000000000 P bde73cb38c8545419aa9d2784e6f4621 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:36.183004 879 sys_catalog.cc:455] T 00000000000000000000000000000000 P bde73cb38c8545419aa9d2784e6f4621 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bde73cb38c8545419aa9d2784e6f4621. Latest consensus state: current_term: 1 leader_uuid: "bde73cb38c8545419aa9d2784e6f4621" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bde73cb38c8545419aa9d2784e6f4621" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36557 } } }
I20260808 03:51:36.183389 879 sys_catalog.cc:458] T 00000000000000000000000000000000 P bde73cb38c8545419aa9d2784e6f4621 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:36.185030 890 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:51:36.190097 890 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:51:36.198940 890 catalog_manager.cc:1347] Generated new cluster ID: 9e2c8110f0404bf4b4ad290de22cf7e5
I20260808 03:51:36.199036 890 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:51:36.213634 890 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:51:36.214856 890 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:51:36.223219 890 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bde73cb38c8545419aa9d2784e6f4621: Generated new TSK 0
I20260808 03:51:36.224071 890 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
I20260808 03:51:36.290859 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:36557
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:40975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:36.635594 904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:36.635901 904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:36.636049 904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:36.643116 904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:36.643293 904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:36.643510 904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:51:36.653131 904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:40975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:36557
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:36.654415 904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:36.655685 904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:36.668226 912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:36.668651 910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:36.668390 917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:36.669785 904 server_base.cc:1034] running on GCE node
I20260808 03:51:36.670480 904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:36.671427 904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:36.672684 904 hybrid_clock.cc:648] HybridClock initialized: now 1786161096672622 us; error 49 us; skew 500 ppm
I20260808 03:51:36.673017 904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:36 dist-test-slave-jlzn krb5kdc[761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161096, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:36.676126 904 init.cc:377] Logged in from keytab as kudu/127.26.146.129@KRBTEST.COM (short username kudu)
I20260808 03:51:36.677145 904 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:36.677246 904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:36.677515 904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:36.680400 904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "90071de63fd249f9bdef993553ea2345"
format_stamp: "Formatted at 2026-08-08 03:51:36 on dist-test-slave-jlzn"
server_key: "666cd6cfcfb6bb0cdb17e320897102dd"
server_key_iv: "f59bf4806133a3eea3e8b58f803f7eda"
server_key_version: "encryptionkey@0"
I20260808 03:51:36.681156 904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "90071de63fd249f9bdef993553ea2345"
format_stamp: "Formatted at 2026-08-08 03:51:36 on dist-test-slave-jlzn"
server_key: "666cd6cfcfb6bb0cdb17e320897102dd"
server_key_iv: "f59bf4806133a3eea3e8b58f803f7eda"
server_key_version: "encryptionkey@0"
I20260808 03:51:36.686148 904 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:51:36.689481 927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:36.690760 904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 03:51:36.690917 904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "90071de63fd249f9bdef993553ea2345"
format_stamp: "Formatted at 2026-08-08 03:51:36 on dist-test-slave-jlzn"
server_key: "666cd6cfcfb6bb0cdb17e320897102dd"
server_key_iv: "f59bf4806133a3eea3e8b58f803f7eda"
server_key_version: "encryptionkey@0"
I20260808 03:51:36.691118 904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:36.704160 904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:36.707723 904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:36.707998 904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:36.708877 904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:36.710145 904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:36.710232 904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:36.710335 904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:36.710414 904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:36.740911 904 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:37569
I20260808 03:51:36.740957 1094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:37569 every 8 connection(s)
I20260808 03:51:36.741770 904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:51:36.744782 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 904
I20260808 03:51:36.745033 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:51:36.745844 27210 external_mini_cluster.cc:1428] Setting key 4c46fce5e59c9126f13dc90aa35b28f7
Aug 08 03:51:36 dist-test-slave-jlzn krb5kdc[761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161096, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:36.760627 1100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:36.744071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:46515 (local address 127.26.146.190:36557)
0808 03:51:36.744514 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:36.744519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:36.745517 (+ 998us) server_negotiation.cc:407] Connection header received
0808 03:51:36.747434 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:36.747439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:36.747525 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:36.747620 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:36.749145 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:36.749842 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:36.751355 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:36.751765 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:36.751927 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:36.755288 (+ 3361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:36.755293 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:36.755296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:36.755327 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:36.757729 (+ 2402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:36.758408 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:36.758413 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:36.758416 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:36.758552 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:36.758946 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:36.758951 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:36.758954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:36.759220 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:36.759350 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:36.760056 (+ 706us) server_negotiation.cc:299] Negotiation successful
0808 03:51:36.760275 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":240,"threads_started":1}
I20260808 03:51:36.761585 1102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:36.744600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36557 (local address 127.26.146.129:46515)
0808 03:51:36.745368 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:36.745405 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:36.747182 (+ 1777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:36.747916 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:36.747927 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:36.748426 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:36.748980 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:36.748998 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:36.749993 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:36.749997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:36.751211 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:36.751224 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:36.752026 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:36.752034 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:36.752259 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:36.752864 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:36.752891 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:36.755089 (+ 2198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:36.757855 (+ 2766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:36.757864 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:36.757880 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:36.758298 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:36.758654 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:36.758660 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:36.758664 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:36.758840 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:36.759340 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:36.759347 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:36.759661 (+ 314us) client_negotiation.cc:769] Sending connection context
0808 03:51:36.759939 (+ 278us) client_negotiation.cc:240] Negotiation successful
0808 03:51:36.760445 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":307,"threads_started":1}
I20260808 03:51:36.763078 1095 heartbeater.cc:347] Connected to a master server at 127.26.146.190:36557
I20260808 03:51:36.763396 1095 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:36.764376 1095 heartbeater.cc:511] Master 127.26.146.190:36557 requested a full tablet report, sending...
I20260808 03:51:36.766638 824 ts_manager.cc:194] Registered new tserver with Master: 90071de63fd249f9bdef993553ea2345 (127.26.146.129:37569)
I20260808 03:51:36.768086 824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.129@KRBTEST.COM'} at 127.26.146.129:46515
WARNING: no policy specified for kudu/127.26.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
I20260808 03:51:36.794674 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:36557
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:40975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:37.113788 1106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:37.114068 1106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:37.114214 1106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:37.121641 1106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:37.121762 1106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:37.121912 1106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:51:37.131464 1106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:40975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:36557
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:37.132799 1106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:37.134057 1106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:37.146596 1116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:37.146674 1114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:37.147228 1113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:37.147759 1106 server_base.cc:1034] running on GCE node
I20260808 03:51:37.148386 1106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:37.149310 1106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:37.150516 1106 hybrid_clock.cc:648] HybridClock initialized: now 1786161097150463 us; error 43 us; skew 500 ppm
I20260808 03:51:37.150822 1106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:37 dist-test-slave-jlzn krb5kdc[761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161097, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:37.156606 1106 init.cc:377] Logged in from keytab as kudu/127.26.146.130@KRBTEST.COM (short username kudu)
I20260808 03:51:37.157743 1106 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:37.157848 1106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:37.158133 1106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:37.161034 1106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "05899ad6796c426bbb17fdcc27b20d6e"
format_stamp: "Formatted at 2026-08-08 03:51:37 on dist-test-slave-jlzn"
server_key: "99c1a7ed3ee71dd8f88a9293a63f8daf"
server_key_iv: "60b2ceb509016fb311f2e92bd0422ce6"
server_key_version: "encryptionkey@0"
I20260808 03:51:37.161789 1106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "05899ad6796c426bbb17fdcc27b20d6e"
format_stamp: "Formatted at 2026-08-08 03:51:37 on dist-test-slave-jlzn"
server_key: "99c1a7ed3ee71dd8f88a9293a63f8daf"
server_key_iv: "60b2ceb509016fb311f2e92bd0422ce6"
server_key_version: "encryptionkey@0"
I20260808 03:51:37.166558 1106 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 03:51:37.169756 1123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:37.170893 1106 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 03:51:37.171047 1106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "05899ad6796c426bbb17fdcc27b20d6e"
format_stamp: "Formatted at 2026-08-08 03:51:37 on dist-test-slave-jlzn"
server_key: "99c1a7ed3ee71dd8f88a9293a63f8daf"
server_key_iv: "60b2ceb509016fb311f2e92bd0422ce6"
server_key_version: "encryptionkey@0"
I20260808 03:51:37.171241 1106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:37.187750 1106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:37.191854 1106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:37.192097 1106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:37.192945 1106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:37.194239 1106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:37.194335 1106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:37.194434 1106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:37.194506 1106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:37.223282 1106 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:39511
I20260808 03:51:37.223330 1286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:39511 every 8 connection(s)
I20260808 03:51:37.224133 1106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:51:37.234236 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 1106
I20260808 03:51:37.234411 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:51:37.234830 27210 external_mini_cluster.cc:1428] Setting key b3eb8dc714cd37f2d2a0b8b98c15a785
Aug 08 03:51:37 dist-test-slave-jlzn krb5kdc[761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161097, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:37.242439 1100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:37.226453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:50679 (local address 127.26.146.190:36557)
0808 03:51:37.226615 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:37.226620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:37.227721 (+ 1101us) server_negotiation.cc:407] Connection header received
0808 03:51:37.229432 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:37.229436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:37.229506 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:37.229583 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:37.230951 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.231660 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.233176 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.233599 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.233700 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:37.237103 (+ 3403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:37.237109 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:37.237112 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:37.237145 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:37.239519 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:37.240261 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:37.240265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:37.240267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:37.240399 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:37.240803 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:37.240806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:37.240808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:37.241069 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:37.241213 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:37.241886 (+ 673us) server_negotiation.cc:299] Negotiation successful
0808 03:51:37.242111 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 03:51:37.243412 1289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:37.226892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36557 (local address 127.26.146.130:50679)
0808 03:51:37.227564 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:37.227600 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:37.229181 (+ 1581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:37.229766 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:37.229773 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:37.230253 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:37.230801 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:37.230824 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.231796 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.231841 (+ 45us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:37.233066 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:37.233078 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.233779 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.233783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:37.233964 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:37.234605 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:37.234626 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:37.236891 (+ 2265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:37.239658 (+ 2767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:37.239667 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:37.239683 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:37.240142 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:37.240504 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:37.240509 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:37.240513 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:37.240689 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:37.241204 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:37.241211 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:37.241526 (+ 315us) client_negotiation.cc:769] Sending connection context
0808 03:51:37.241755 (+ 229us) client_negotiation.cc:240] Negotiation successful
0808 03:51:37.242272 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":276,"threads_started":1}
I20260808 03:51:37.244721 1287 heartbeater.cc:347] Connected to a master server at 127.26.146.190:36557
I20260808 03:51:37.244993 1287 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:37.245599 1287 heartbeater.cc:511] Master 127.26.146.190:36557 requested a full tablet report, sending...
I20260808 03:51:37.246956 824 ts_manager.cc:194] Registered new tserver with Master: 05899ad6796c426bbb17fdcc27b20d6e (127.26.146.130:39511)
I20260808 03:51:37.248054 824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.130@KRBTEST.COM'} at 127.26.146.130:50679
WARNING: no policy specified for kudu/127.26.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
I20260808 03:51:37.281167 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:36557
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:40975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:37.596701 1299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:37.596961 1299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:37.597095 1299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:37.604197 1299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:37.604297 1299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:37.604444 1299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:51:37.613741 1299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:40975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:36557
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:37.614926 1299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:37.616240 1299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:37.628613 1305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:37.628592 1308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:37.628675 1310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:37.629341 1299 server_base.cc:1034] running on GCE node
I20260808 03:51:37.629896 1299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:37.630739 1299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:37.631893 1299 hybrid_clock.cc:648] HybridClock initialized: now 1786161097631839 us; error 44 us; skew 500 ppm
I20260808 03:51:37.632206 1299 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:37 dist-test-slave-jlzn krb5kdc[761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161097, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:37.635155 1299 init.cc:377] Logged in from keytab as kudu/127.26.146.131@KRBTEST.COM (short username kudu)
I20260808 03:51:37.636267 1299 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:37.636358 1299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:37.636690 1299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:37.639430 1299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "e531036568604868a55e46e83f29365d"
format_stamp: "Formatted at 2026-08-08 03:51:37 on dist-test-slave-jlzn"
server_key: "9c2b03ebfd7e0dd6c89643a03d64f7d0"
server_key_iv: "215425b219872d97187a829d8f4ba89f"
server_key_version: "encryptionkey@0"
I20260808 03:51:37.640235 1299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "e531036568604868a55e46e83f29365d"
format_stamp: "Formatted at 2026-08-08 03:51:37 on dist-test-slave-jlzn"
server_key: "9c2b03ebfd7e0dd6c89643a03d64f7d0"
server_key_iv: "215425b219872d97187a829d8f4ba89f"
server_key_version: "encryptionkey@0"
I20260808 03:51:37.644961 1299 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 03:51:37.648222 1317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:37.649444 1299 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:51:37.649591 1299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "e531036568604868a55e46e83f29365d"
format_stamp: "Formatted at 2026-08-08 03:51:37 on dist-test-slave-jlzn"
server_key: "9c2b03ebfd7e0dd6c89643a03d64f7d0"
server_key_iv: "215425b219872d97187a829d8f4ba89f"
server_key_version: "encryptionkey@0"
I20260808 03:51:37.649793 1299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:37.667965 1299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:37.672364 1299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:37.672636 1299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:37.673800 1299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:37.675179 1299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:37.675264 1299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:37.675344 1299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:37.675410 1299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:37.706141 1299 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:40907
I20260808 03:51:37.706183 1477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:40907 every 8 connection(s)
I20260808 03:51:37.706929 1299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:51:37.709535 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 1299
I20260808 03:51:37.709743 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:51:37.710310 27210 external_mini_cluster.cc:1428] Setting key b60129c1d75427fce2bc698a174eddfa
Aug 08 03:51:37 dist-test-slave-jlzn krb5kdc[761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161097, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:37.726258 1100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:37.709171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:56463 (local address 127.26.146.190:36557)
0808 03:51:37.709331 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:37.709336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:37.710681 (+ 1345us) server_negotiation.cc:407] Connection header received
0808 03:51:37.713269 (+ 2588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:37.713274 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:37.713341 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:37.713443 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:37.714999 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.715755 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.717254 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.717668 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.717757 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:37.720930 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:37.720936 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:37.720940 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:37.720973 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:37.723229 (+ 2256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:37.723909 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:37.723914 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:37.723917 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:37.724073 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:37.724524 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:37.724527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:37.724529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:37.724872 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:37.725021 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:37.725796 (+ 775us) server_negotiation.cc:299] Negotiation successful
0808 03:51:37.725997 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 03:51:37.727115 1483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:37.709735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36557 (local address 127.26.146.131:56463)
0808 03:51:37.710522 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:37.710568 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:37.712878 (+ 2310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:37.713715 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:37.713723 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:37.714229 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:37.714847 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:37.714865 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.715932 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.715936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:37.717128 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:37.717140 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.717828 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.717832 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:37.718009 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:37.718612 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:37.718634 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:37.720720 (+ 2086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:37.723351 (+ 2631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:37.723358 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:37.723368 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:37.723771 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:37.724192 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:37.724196 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:37.724199 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:37.724411 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:37.725003 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:37.725009 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:37.725374 (+ 365us) client_negotiation.cc:769] Sending connection context
0808 03:51:37.725664 (+ 290us) client_negotiation.cc:240] Negotiation successful
0808 03:51:37.726103 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":318,"threads_started":1}
I20260808 03:51:37.728367 1479 heartbeater.cc:347] Connected to a master server at 127.26.146.190:36557
I20260808 03:51:37.728632 1479 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:37.729228 1479 heartbeater.cc:511] Master 127.26.146.190:36557 requested a full tablet report, sending...
I20260808 03:51:37.730461 824 ts_manager.cc:194] Registered new tserver with Master: e531036568604868a55e46e83f29365d (127.26.146.131:40907)
I20260808 03:51:37.731226 824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.131@KRBTEST.COM'} at 127.26.146.131:56463
I20260808 03:51:37.737721 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:51:37.754585 1100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:37.742306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51228 (local address 127.26.146.190:36557)
0808 03:51:37.742461 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:37.742465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:37.742989 (+ 524us) server_negotiation.cc:407] Connection header received
0808 03:51:37.743161 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:37.743165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:37.743222 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:37.743297 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:37.744910 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.745634 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.746875 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.747299 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.747405 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:37.749522 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:37.749527 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:37.749530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:37.749560 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:37.751836 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:37.752502 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:37.752521 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:37.752524 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:37.752661 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:37.753036 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:37.753041 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:37.753045 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:37.753361 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:37.753639 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:37.754048 (+ 409us) server_negotiation.cc:299] Negotiation successful
0808 03:51:37.754259 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 03:51:37.758250 824 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51228:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:51:37.764945 824 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:51:37.770910 1095 heartbeater.cc:503] Master 127.26.146.190:36557 was elected leader, sending a full tablet report...
I20260808 03:51:37.786772 1504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:37.777178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58362 (local address 127.26.146.129:37569)
0808 03:51:37.778894 (+ 1716us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:37.778900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:37.779547 (+ 647us) server_negotiation.cc:407] Connection header received
0808 03:51:37.779660 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:37.779667 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:37.779904 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:37.780083 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:37.781006 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.782519 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.784599 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.785613 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.785798 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:37.785967 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:37.786289 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 03:51:37.786468 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1591,"thread_start_us":1419,"threads_started":1}
I20260808 03:51:37.786891 1498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:37.776704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:37569 (local address 127.0.0.1:58362)
0808 03:51:37.779257 (+ 2553us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:37.779275 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:37.779410 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:37.780315 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:37.780320 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:37.780346 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:37.780819 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:37.780833 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.782662 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.782668 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:37.784464 (+ 1796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:37.784480 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.785820 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.785826 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:37.786072 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:37.786108 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 03:51:37.786216 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 03:51:37.786403 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2411,"mutex_wait_us":30,"thread_start_us":627,"threads_started":1}
I20260808 03:51:37.787268 1503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:37.776313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:40907 (local address 127.0.0.1:58554)
0808 03:51:37.777747 (+ 1434us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:37.777793 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:37.777969 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:37.779301 (+ 1332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:37.779309 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:37.779340 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:37.779846 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:37.779863 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.782359 (+ 2496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.782364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:37.784737 (+ 2373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:37.784752 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.786501 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.786508 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:37.786705 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:37.786738 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 03:51:37.786795 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 03:51:37.786925 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1151,"thread_start_us":2689,"threads_started":1}
I20260808 03:51:37.788587 1500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:37.776444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58554 (local address 127.26.146.131:40907)
0808 03:51:37.778641 (+ 2197us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:37.778651 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:37.778686 (+ 35us) server_negotiation.cc:407] Connection header received
0808 03:51:37.778785 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:37.778793 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:37.779033 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:37.780078 (+ 1045us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:37.780467 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.782185 (+ 1718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.784887 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.786261 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.787882 (+ 1621us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:37.788043 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:37.788108 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 03:51:37.788253 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1995,"thread_start_us":1813,"threads_started":1}
I20260808 03:51:37.789407 1507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:37.780119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:39511 (local address 127.0.0.1:38466)
0808 03:51:37.780959 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:37.780974 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:37.781078 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:37.781732 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:37.781735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:37.781754 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:37.782014 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:37.782020 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.784533 (+ 2513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.784538 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:37.786586 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:37.786599 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.788657 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.788664 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:37.788815 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:37.788837 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 03:51:37.788917 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 03:51:37.789033 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"mutex_wait_us":26,"thread_start_us":572,"threads_started":1}
I20260808 03:51:37.789440 1505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:37.779645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38466 (local address 127.26.146.130:39511)
0808 03:51:37.780534 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:37.780541 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:37.781049 (+ 508us) server_negotiation.cc:407] Connection header received
0808 03:51:37.781226 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:37.781232 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:37.781400 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:37.781553 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:37.782189 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.784401 (+ 2212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.786787 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.788194 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.788335 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:37.788483 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:37.788989 (+ 506us) server_negotiation.cc:299] Negotiation successful
0808 03:51:37.789142 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":740,"thread_start_us":594,"threads_started":1}
I20260808 03:51:37.790691 999 tablet_service.cc:1467] Processing CreateTablet for tablet 91900bb1e52f461fa4fc73a7fa8bc2b5 (DEFAULT_TABLE table=test-table [id=7e70bfb140874608be84c7dcd06dc74c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:37.791939 999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91900bb1e52f461fa4fc73a7fa8bc2b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:37.792284 1194 tablet_service.cc:1467] Processing CreateTablet for tablet 91900bb1e52f461fa4fc73a7fa8bc2b5 (DEFAULT_TABLE table=test-table [id=7e70bfb140874608be84c7dcd06dc74c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:37.793592 1194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91900bb1e52f461fa4fc73a7fa8bc2b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:37.795244 1392 tablet_service.cc:1467] Processing CreateTablet for tablet 91900bb1e52f461fa4fc73a7fa8bc2b5 (DEFAULT_TABLE table=test-table [id=7e70bfb140874608be84c7dcd06dc74c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:37.796542 1392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91900bb1e52f461fa4fc73a7fa8bc2b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:37.803709 1508 tablet_bootstrap.cc:492] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 90071de63fd249f9bdef993553ea2345: Bootstrap starting.
I20260808 03:51:37.807091 1508 tablet_bootstrap.cc:654] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 90071de63fd249f9bdef993553ea2345: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:37.807431 1509 tablet_bootstrap.cc:492] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e: Bootstrap starting.
I20260808 03:51:37.808734 1508 log.cc:824] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 90071de63fd249f9bdef993553ea2345: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:37.809617 1510 tablet_bootstrap.cc:492] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P e531036568604868a55e46e83f29365d: Bootstrap starting.
I20260808 03:51:37.810899 1509 tablet_bootstrap.cc:654] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:37.812161 1509 log.cc:824] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:37.812462 1510 tablet_bootstrap.cc:654] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P e531036568604868a55e46e83f29365d: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:37.813422 1510 log.cc:824] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P e531036568604868a55e46e83f29365d: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:37.814549 1508 tablet_bootstrap.cc:492] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 90071de63fd249f9bdef993553ea2345: No bootstrap required, opened a new log
I20260808 03:51:37.814841 1508 ts_tablet_manager.cc:1397] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 90071de63fd249f9bdef993553ea2345: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260808 03:51:37.821576 1509 tablet_bootstrap.cc:492] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e: No bootstrap required, opened a new log
I20260808 03:51:37.821871 1510 tablet_bootstrap.cc:492] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P e531036568604868a55e46e83f29365d: No bootstrap required, opened a new log
I20260808 03:51:37.821870 1509 ts_tablet_manager.cc:1397] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20260808 03:51:37.822139 1510 ts_tablet_manager.cc:1397] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P e531036568604868a55e46e83f29365d: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.005s
I20260808 03:51:37.823881 1508 raft_consensus.cc:348] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 90071de63fd249f9bdef993553ea2345 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e531036568604868a55e46e83f29365d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 40907 } } peers { permanent_uuid: "05899ad6796c426bbb17fdcc27b20d6e" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 39511 } } peers { permanent_uuid: "90071de63fd249f9bdef993553ea2345" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37569 } }
I20260808 03:51:37.824314 1508 raft_consensus.cc:374] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 90071de63fd249f9bdef993553ea2345 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:37.824411 1508 raft_consensus.cc:729] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 90071de63fd249f9bdef993553ea2345 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90071de63fd249f9bdef993553ea2345, State: Initialized, Role: FOLLOWER
I20260808 03:51:37.825121 1508 consensus_queue.cc:260] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 90071de63fd249f9bdef993553ea2345 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e531036568604868a55e46e83f29365d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 40907 } } peers { permanent_uuid: "05899ad6796c426bbb17fdcc27b20d6e" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 39511 } } peers { permanent_uuid: "90071de63fd249f9bdef993553ea2345" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37569 } }
I20260808 03:51:37.826786 1508 ts_tablet_manager.cc:1428] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 90071de63fd249f9bdef993553ea2345: Time spent starting tablet: real 0.012s user 0.006s sys 0.005s
I20260808 03:51:37.832088 1509 raft_consensus.cc:348] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e531036568604868a55e46e83f29365d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 40907 } } peers { permanent_uuid: "05899ad6796c426bbb17fdcc27b20d6e" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 39511 } } peers { permanent_uuid: "90071de63fd249f9bdef993553ea2345" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37569 } }
I20260808 03:51:37.832088 1510 raft_consensus.cc:348] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P e531036568604868a55e46e83f29365d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e531036568604868a55e46e83f29365d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 40907 } } peers { permanent_uuid: "05899ad6796c426bbb17fdcc27b20d6e" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 39511 } } peers { permanent_uuid: "90071de63fd249f9bdef993553ea2345" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37569 } }
I20260808 03:51:37.832513 1510 raft_consensus.cc:374] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P e531036568604868a55e46e83f29365d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:37.832513 1509 raft_consensus.cc:374] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:37.832620 1510 raft_consensus.cc:729] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P e531036568604868a55e46e83f29365d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e531036568604868a55e46e83f29365d, State: Initialized, Role: FOLLOWER
I20260808 03:51:37.832630 1509 raft_consensus.cc:729] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05899ad6796c426bbb17fdcc27b20d6e, State: Initialized, Role: FOLLOWER
I20260808 03:51:37.833202 1510 consensus_queue.cc:260] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P e531036568604868a55e46e83f29365d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e531036568604868a55e46e83f29365d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 40907 } } peers { permanent_uuid: "05899ad6796c426bbb17fdcc27b20d6e" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 39511 } } peers { permanent_uuid: "90071de63fd249f9bdef993553ea2345" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37569 } }
I20260808 03:51:37.833200 1509 consensus_queue.cc:260] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e531036568604868a55e46e83f29365d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 40907 } } peers { permanent_uuid: "05899ad6796c426bbb17fdcc27b20d6e" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 39511 } } peers { permanent_uuid: "90071de63fd249f9bdef993553ea2345" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37569 } }
I20260808 03:51:37.834270 1287 heartbeater.cc:503] Master 127.26.146.190:36557 was elected leader, sending a full tablet report...
I20260808 03:51:37.834280 1479 heartbeater.cc:503] Master 127.26.146.190:36557 was elected leader, sending a full tablet report...
I20260808 03:51:37.834918 1509 ts_tablet_manager.cc:1428] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260808 03:51:37.834910 1510 ts_tablet_manager.cc:1428] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P e531036568604868a55e46e83f29365d: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260808 03:51:37.927863 1518 raft_consensus.cc:482] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:51:37.928171 1518 raft_consensus.cc:504] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e531036568604868a55e46e83f29365d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 40907 } } peers { permanent_uuid: "05899ad6796c426bbb17fdcc27b20d6e" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 39511 } } peers { permanent_uuid: "90071de63fd249f9bdef993553ea2345" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37569 } }
I20260808 03:51:37.929943 1518 leader_election.cc:288] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e531036568604868a55e46e83f29365d (127.26.146.131:40907), 90071de63fd249f9bdef993553ea2345 (127.26.146.129:37569)
I20260808 03:51:37.938344 1500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:37.930296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:50101 (local address 127.26.146.131:40907)
0808 03:51:37.930501 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:37.930508 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:37.931041 (+ 533us) server_negotiation.cc:407] Connection header received
0808 03:51:37.931294 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:37.931299 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:37.931379 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:37.931477 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:37.932258 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.933357 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.936150 (+ 2793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.937349 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.937484 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:37.937554 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:37.937968 (+ 414us) server_negotiation.cc:299] Negotiation successful
0808 03:51:37.938090 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 03:51:37.938349 1504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:37.930591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:36433 (local address 127.26.146.129:37569)
0808 03:51:37.930827 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:37.930834 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:37.931204 (+ 370us) server_negotiation.cc:407] Connection header received
0808 03:51:37.931354 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:37.931359 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:37.931433 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:37.931525 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:37.932262 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.933357 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.936150 (+ 2793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.937319 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.937482 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:37.937554 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:37.937968 (+ 414us) server_negotiation.cc:299] Negotiation successful
0808 03:51:37.938090 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 03:51:37.938616 1522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:37.930399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:37569 (local address 127.26.146.130:36433)
0808 03:51:37.931112 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:37.931126 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:37.931229 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:37.931547 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:37.931551 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:37.931605 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:37.932125 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:37.932135 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.933477 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.933481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:37.936008 (+ 2527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:37.936024 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.937561 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.937567 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:37.937775 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:37.937845 (+ 70us) client_negotiation.cc:769] Sending connection context
0808 03:51:37.937939 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 03:51:37.938127 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"mutex_wait_us":172,"spinlock_wait_cycles":6272,"thread_start_us":426,"threads_started":1}
I20260808 03:51:37.938642 1520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:37.930139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:40907 (local address 127.26.146.130:50101)
0808 03:51:37.930903 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:37.930931 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:37.931095 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:37.931533 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:37.931538 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:37.931603 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:37.932099 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:37.932117 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.933473 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.933479 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:37.936008 (+ 2529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:37.936024 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.937566 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.937571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:37.937779 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:37.937843 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 03:51:37.937938 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 03:51:37.938132 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"mutex_wait_us":191,"spinlock_wait_cycles":58368,"thread_start_us":373,"threads_started":1}
I20260808 03:51:37.939558 1417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91900bb1e52f461fa4fc73a7fa8bc2b5" candidate_uuid: "05899ad6796c426bbb17fdcc27b20d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e531036568604868a55e46e83f29365d" is_pre_election: true
I20260808 03:51:37.939704 1031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91900bb1e52f461fa4fc73a7fa8bc2b5" candidate_uuid: "05899ad6796c426bbb17fdcc27b20d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90071de63fd249f9bdef993553ea2345" is_pre_election: true
I20260808 03:51:37.940099 1417 raft_consensus.cc:2454] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P e531036568604868a55e46e83f29365d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05899ad6796c426bbb17fdcc27b20d6e in term 0.
I20260808 03:51:37.940209 1031 raft_consensus.cc:2454] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 90071de63fd249f9bdef993553ea2345 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05899ad6796c426bbb17fdcc27b20d6e in term 0.
I20260808 03:51:37.940778 1124 leader_election.cc:302] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05899ad6796c426bbb17fdcc27b20d6e, e531036568604868a55e46e83f29365d; no voters:
I20260808 03:51:37.941123 1518 raft_consensus.cc:2781] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:51:37.941274 1518 raft_consensus.cc:482] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:51:37.941376 1518 raft_consensus.cc:3037] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:37.943634 1518 raft_consensus.cc:504] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e531036568604868a55e46e83f29365d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 40907 } } peers { permanent_uuid: "05899ad6796c426bbb17fdcc27b20d6e" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 39511 } } peers { permanent_uuid: "90071de63fd249f9bdef993553ea2345" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37569 } }
I20260808 03:51:37.944242 1518 leader_election.cc:288] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e [CANDIDATE]: Term 1 election: Requested vote from peers e531036568604868a55e46e83f29365d (127.26.146.131:40907), 90071de63fd249f9bdef993553ea2345 (127.26.146.129:37569)
I20260808 03:51:37.944563 1417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91900bb1e52f461fa4fc73a7fa8bc2b5" candidate_uuid: "05899ad6796c426bbb17fdcc27b20d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e531036568604868a55e46e83f29365d"
I20260808 03:51:37.944711 1031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91900bb1e52f461fa4fc73a7fa8bc2b5" candidate_uuid: "05899ad6796c426bbb17fdcc27b20d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90071de63fd249f9bdef993553ea2345"
I20260808 03:51:37.944763 1417 raft_consensus.cc:3037] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P e531036568604868a55e46e83f29365d [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:37.944907 1031 raft_consensus.cc:3037] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 90071de63fd249f9bdef993553ea2345 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:37.947927 1031 raft_consensus.cc:2454] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 90071de63fd249f9bdef993553ea2345 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05899ad6796c426bbb17fdcc27b20d6e in term 1.
I20260808 03:51:37.947927 1417 raft_consensus.cc:2454] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P e531036568604868a55e46e83f29365d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05899ad6796c426bbb17fdcc27b20d6e in term 1.
I20260808 03:51:37.948302 1124 leader_election.cc:302] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 05899ad6796c426bbb17fdcc27b20d6e, 90071de63fd249f9bdef993553ea2345, e531036568604868a55e46e83f29365d; no voters:
I20260808 03:51:37.948606 1518 raft_consensus.cc:2781] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:37.949007 1518 raft_consensus.cc:686] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e [term 1 LEADER]: Becoming Leader. State: Replica: 05899ad6796c426bbb17fdcc27b20d6e, State: Running, Role: LEADER
I20260808 03:51:37.949466 1518 consensus_queue.cc:237] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e531036568604868a55e46e83f29365d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 40907 } } peers { permanent_uuid: "05899ad6796c426bbb17fdcc27b20d6e" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 39511 } } peers { permanent_uuid: "90071de63fd249f9bdef993553ea2345" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37569 } }
I20260808 03:51:37.953706 824 catalog_manager.cc:5495] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 05899ad6796c426bbb17fdcc27b20d6e (127.26.146.130). New cstate: current_term: 1 leader_uuid: "05899ad6796c426bbb17fdcc27b20d6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e531036568604868a55e46e83f29365d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 40907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05899ad6796c426bbb17fdcc27b20d6e" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 39511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90071de63fd249f9bdef993553ea2345" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37569 } health_report { overall_health: UNKNOWN } } }
W20260808 03:51:37.958257 1482 tablet.cc:2369] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P e531036568604868a55e46e83f29365d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:51:37.974912 1505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:37.970288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38478 (local address 127.26.146.130:39511)
0808 03:51:37.970452 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:37.970457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:37.970474 (+ 17us) server_negotiation.cc:407] Connection header received
0808 03:51:37.970584 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:37.970588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:37.970667 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:37.970781 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:51:37.971069 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.971787 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.973051 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:37.973446 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:37.973529 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:37.973888 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:51:37.974419 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:51:37.974520 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:37.974599 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 03:51:37.974661 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 03:51:37.975764 1288 tablet.cc:2369] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:51:37.982489 1417 raft_consensus.cc:1261] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P e531036568604868a55e46e83f29365d [term 1 FOLLOWER]: Refusing update from remote peer 05899ad6796c426bbb17fdcc27b20d6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:51:37.982493 1031 raft_consensus.cc:1261] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 90071de63fd249f9bdef993553ea2345 [term 1 FOLLOWER]: Refusing update from remote peer 05899ad6796c426bbb17fdcc27b20d6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:51:37.983422 1518 consensus_queue.cc:1035] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "90071de63fd249f9bdef993553ea2345" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:51:37.983690 1523 consensus_queue.cc:1035] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e531036568604868a55e46e83f29365d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 40907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 03:51:37.994383 1098 tablet.cc:2369] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 90071de63fd249f9bdef993553ea2345: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:51:38.008574 823 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:51:38.008822 823 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:51:38.012460 823 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bde73cb38c8545419aa9d2784e6f4621: Sending DeleteTablet for 3 replicas of tablet 91900bb1e52f461fa4fc73a7fa8bc2b5
I20260808 03:51:38.013566 1194 tablet_service.cc:1514] Processing DeleteTablet for tablet 91900bb1e52f461fa4fc73a7fa8bc2b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:38 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:38466
I20260808 03:51:38.013554 999 tablet_service.cc:1514] Processing DeleteTablet for tablet 91900bb1e52f461fa4fc73a7fa8bc2b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:38 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:58362
I20260808 03:51:38.013607 1392 tablet_service.cc:1514] Processing DeleteTablet for tablet 91900bb1e52f461fa4fc73a7fa8bc2b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:38 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:58554
I20260808 03:51:38.015589 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 904
I20260808 03:51:38.026353 1544 tablet_replica.cc:330] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 90071de63fd249f9bdef993553ea2345: stopping tablet replica
I20260808 03:51:38.026934 1544 raft_consensus.cc:2229] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 90071de63fd249f9bdef993553ea2345 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:51:38.027508 1544 raft_consensus.cc:2258] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 90071de63fd249f9bdef993553ea2345 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:38.028520 1543 tablet_replica.cc:330] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e: stopping tablet replica
I20260808 03:51:38.030038 1544 ts_tablet_manager.cc:1905] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 90071de63fd249f9bdef993553ea2345: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:38.037112 1544 ts_tablet_manager.cc:1918] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 90071de63fd249f9bdef993553ea2345: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:51:38.037647 1543 raft_consensus.cc:2229] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:51:38.038501 1543 raft_consensus.cc:2258] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:38.039263 1544 log.cc:1196] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 90071de63fd249f9bdef993553ea2345: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-0/wal/wals/91900bb1e52f461fa4fc73a7fa8bc2b5
I20260808 03:51:38.039388 1545 tablet_replica.cc:330] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P e531036568604868a55e46e83f29365d: stopping tablet replica
I20260808 03:51:38.039798 1545 raft_consensus.cc:2229] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P e531036568604868a55e46e83f29365d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:51:38.039934 1544 ts_tablet_manager.cc:1939] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 90071de63fd249f9bdef993553ea2345: Deleting consensus metadata
I20260808 03:51:38.040387 1545 raft_consensus.cc:2258] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P e531036568604868a55e46e83f29365d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:38.041494 1543 ts_tablet_manager.cc:1905] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:38.041859 804 catalog_manager.cc:4841] TS 90071de63fd249f9bdef993553ea2345 (127.26.146.129:37569): tablet 91900bb1e52f461fa4fc73a7fa8bc2b5 (table test-table [id=7e70bfb140874608be84c7dcd06dc74c]) successfully deleted
Aug 08 03:51:38 dist-test-slave-jlzn krb5kdc[761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.129@KRBTEST.COM
I20260808 03:51:38.043696 1545 ts_tablet_manager.cc:1905] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P e531036568604868a55e46e83f29365d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:38.049330 1543 ts_tablet_manager.cc:1918] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:51:38.049513 1543 log.cc:1196] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/91900bb1e52f461fa4fc73a7fa8bc2b5
I20260808 03:51:38.050138 1543 ts_tablet_manager.cc:1939] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P 05899ad6796c426bbb17fdcc27b20d6e: Deleting consensus metadata
I20260808 03:51:38.050736 1545 ts_tablet_manager.cc:1918] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P e531036568604868a55e46e83f29365d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:51:38.050945 1545 log.cc:1196] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P e531036568604868a55e46e83f29365d: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/91900bb1e52f461fa4fc73a7fa8bc2b5
I20260808 03:51:38.051553 1545 ts_tablet_manager.cc:1939] T 91900bb1e52f461fa4fc73a7fa8bc2b5 P e531036568604868a55e46e83f29365d: Deleting consensus metadata
I20260808 03:51:38.051937 802 catalog_manager.cc:4841] TS 05899ad6796c426bbb17fdcc27b20d6e (127.26.146.130:39511): tablet 91900bb1e52f461fa4fc73a7fa8bc2b5 (table test-table [id=7e70bfb140874608be84c7dcd06dc74c]) successfully deleted
I20260808 03:51:38.052269 1504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:38.017219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58370 (local address 127.26.146.129:37569)
0808 03:51:38.017397 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:38.017404 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:38.032179 (+ 14775us) server_negotiation.cc:407] Connection header received
0808 03:51:38.032612 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:38.032618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:38.032699 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:38.032771 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:38.034526 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:38.035714 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:38.037517 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:38.038067 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:38.038173 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:38.044866 (+ 6693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:38.044879 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:38.044897 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:38.044937 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:38.048575 (+ 3638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:38.049445 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:38.049454 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:38.049457 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:38.049633 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:38.050105 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:38.050110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:38.050113 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:38.050592 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:38.050895 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:38.051513 (+ 618us) server_negotiation.cc:299] Negotiation successful
0808 03:51:38.051911 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 03:51:38.053308 1090 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:38.053629 802 catalog_manager.cc:4841] TS e531036568604868a55e46e83f29365d (127.26.146.131:40907): tablet 91900bb1e52f461fa4fc73a7fa8bc2b5 (table test-table [id=7e70bfb140874608be84c7dcd06dc74c]) successfully deleted
I20260808 03:51:38.144806 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 904
I20260808 03:51:38.170091 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 1106
Aug 08 03:51:38 dist-test-slave-jlzn krb5kdc[761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.130@KRBTEST.COM
I20260808 03:51:38.185678 1505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:38.170847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38488 (local address 127.26.146.130:39511)
0808 03:51:38.171022 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:38.171028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:38.171104 (+ 76us) server_negotiation.cc:407] Connection header received
0808 03:51:38.171298 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:38.171304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:38.171374 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:38.171464 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:38.172735 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:38.173544 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:38.174837 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:38.175434 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:38.175591 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:38.179623 (+ 4032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:38.179635 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:38.179651 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:38.179686 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:38.182811 (+ 3125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:38.183533 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:38.183543 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:38.183546 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:38.183671 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:38.184160 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:38.184164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:38.184166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:38.184520 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:38.184836 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:38.185116 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 03:51:38.185341 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 03:51:38.186299 1281 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:38.299199 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 1106
I20260808 03:51:38.322160 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 1299
Aug 08 03:51:38 dist-test-slave-jlzn krb5kdc[761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.131@KRBTEST.COM
I20260808 03:51:38.340737 1500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:38.322957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58570 (local address 127.26.146.131:40907)
0808 03:51:38.323153 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:38.323157 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:38.323178 (+ 21us) server_negotiation.cc:407] Connection header received
0808 03:51:38.323321 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:38.323325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:38.323398 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:38.323480 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:38.324835 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:38.325574 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:38.326690 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:38.327062 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:38.327148 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:38.335478 (+ 8330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:38.335497 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:38.335520 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:38.335562 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:38.338215 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:38.338845 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:38.338852 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:38.338854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:38.339006 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:38.339370 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:38.339373 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:38.339375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:38.339724 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:38.339973 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:38.340195 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 03:51:38.340391 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 03:51:38.341401 1473 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:38.428856 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 1299
I20260808 03:51:38.450219 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 780
I20260808 03:51:38.451365 865 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:38.558758 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 780
2026-08-08T03:51:38Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4217 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T03:51:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:51:38Z Disabled control of system clock
I20260808 03:51:38.634662 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:42633
--webserver_interface=127.26.146.190
--webserver_port=0
--builtin_ntp_servers=127.26.146.148:46591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:42633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 03:51:38.975554 1555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:38.975878 1555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:38.975972 1555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:38.982991 1555 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:51:38.983093 1555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:38.983139 1555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:38.983196 1555 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:51:38.983240 1555 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:51:38.992223 1555 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:46591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:42633
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:42633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:38.993456 1555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:38.995190 1555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:39.006587 1566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:39.006641 1562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:39.006692 1563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:39.007709 1555 server_base.cc:1034] running on GCE node
I20260808 03:51:39.008508 1555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:39.009807 1555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:39.011001 1555 hybrid_clock.cc:648] HybridClock initialized: now 1786161099010953 us; error 45 us; skew 500 ppm
I20260808 03:51:39.011345 1555 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:51:39.013640 1555 webserver.cc:460] Webserver started at http://127.26.146.190:34767/ using document root <none> and password file <none>
I20260808 03:51:39.014317 1555 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:39.014415 1555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:39.014740 1555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:39.017611 1555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "dc89742e6ce64b56a569e695b8459dcd"
format_stamp: "Formatted at 2026-08-08 03:51:39 on dist-test-slave-jlzn"
server_key: "4a38eeea797bd7a92ce283bc740e1667"
server_key_iv: "e4968a9a14027968b710fd90ee6588f7"
server_key_version: "encryptionkey@0"
I20260808 03:51:39.018395 1555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "dc89742e6ce64b56a569e695b8459dcd"
format_stamp: "Formatted at 2026-08-08 03:51:39 on dist-test-slave-jlzn"
server_key: "4a38eeea797bd7a92ce283bc740e1667"
server_key_iv: "e4968a9a14027968b710fd90ee6588f7"
server_key_version: "encryptionkey@0"
I20260808 03:51:39.023330 1555 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260808 03:51:39.026645 1575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:39.027984 1555 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 03:51:39.028146 1555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "dc89742e6ce64b56a569e695b8459dcd"
format_stamp: "Formatted at 2026-08-08 03:51:39 on dist-test-slave-jlzn"
server_key: "4a38eeea797bd7a92ce283bc740e1667"
server_key_iv: "e4968a9a14027968b710fd90ee6588f7"
server_key_version: "encryptionkey@0"
I20260808 03:51:39.028357 1555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:39.033962 1555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:39.034706 1555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:39.034979 1555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:39.051950 1555 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:42633
I20260808 03:51:39.051981 1642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:42633 every 8 connection(s)
I20260808 03:51:39.053252 1555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:51:39.056855 1643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:39.063673 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 1555
I20260808 03:51:39.063894 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:51:39.064698 27210 external_mini_cluster.cc:1428] Setting key 6012c4c05351fd8306c8a9965e243c4d
I20260808 03:51:39.067796 1643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc89742e6ce64b56a569e695b8459dcd: Bootstrap starting.
I20260808 03:51:39.070943 1643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc89742e6ce64b56a569e695b8459dcd: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:39.072103 1643 log.cc:824] T 00000000000000000000000000000000 P dc89742e6ce64b56a569e695b8459dcd: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:39.073105 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:39.066794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39810 (local address 127.26.146.190:42633)
0808 03:51:39.067725 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:39.067736 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:39.067778 (+ 42us) server_negotiation.cc:407] Connection header received
0808 03:51:39.069479 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:39.069510 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:39.069892 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:39.070235 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:39.071298 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:39.071307 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:39.071320 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:51:39.071346 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:39.071409 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:51:39.071425 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:39.071601 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:39.071742 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 03:51:39.072030 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":316,"threads_started":1}
I20260808 03:51:39.075148 1643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc89742e6ce64b56a569e695b8459dcd: No bootstrap required, opened a new log
I20260808 03:51:39.082273 1643 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc89742e6ce64b56a569e695b8459dcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc89742e6ce64b56a569e695b8459dcd" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 42633 } }
I20260808 03:51:39.082616 1643 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc89742e6ce64b56a569e695b8459dcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:39.082715 1643 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc89742e6ce64b56a569e695b8459dcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc89742e6ce64b56a569e695b8459dcd, State: Initialized, Role: FOLLOWER
I20260808 03:51:39.083227 1643 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc89742e6ce64b56a569e695b8459dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc89742e6ce64b56a569e695b8459dcd" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 42633 } }
I20260808 03:51:39.083436 1643 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc89742e6ce64b56a569e695b8459dcd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:51:39.083559 1643 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc89742e6ce64b56a569e695b8459dcd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:51:39.083739 1643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc89742e6ce64b56a569e695b8459dcd [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:39.085845 1643 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc89742e6ce64b56a569e695b8459dcd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc89742e6ce64b56a569e695b8459dcd" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 42633 } }
I20260808 03:51:39.086302 1643 leader_election.cc:302] T 00000000000000000000000000000000 P dc89742e6ce64b56a569e695b8459dcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc89742e6ce64b56a569e695b8459dcd; no voters:
I20260808 03:51:39.086951 1643 leader_election.cc:288] T 00000000000000000000000000000000 P dc89742e6ce64b56a569e695b8459dcd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:51:39.087069 1649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc89742e6ce64b56a569e695b8459dcd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:39.087841 1649 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc89742e6ce64b56a569e695b8459dcd [term 1 LEADER]: Becoming Leader. State: Replica: dc89742e6ce64b56a569e695b8459dcd, State: Running, Role: LEADER
I20260808 03:51:39.088353 1649 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc89742e6ce64b56a569e695b8459dcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc89742e6ce64b56a569e695b8459dcd" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 42633 } }
I20260808 03:51:39.088995 1643 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc89742e6ce64b56a569e695b8459dcd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:51:39.091029 1651 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc89742e6ce64b56a569e695b8459dcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc89742e6ce64b56a569e695b8459dcd. Latest consensus state: current_term: 1 leader_uuid: "dc89742e6ce64b56a569e695b8459dcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc89742e6ce64b56a569e695b8459dcd" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 42633 } } }
I20260808 03:51:39.091504 1651 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc89742e6ce64b56a569e695b8459dcd [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:39.091290 1650 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc89742e6ce64b56a569e695b8459dcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc89742e6ce64b56a569e695b8459dcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc89742e6ce64b56a569e695b8459dcd" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 42633 } } }
I20260808 03:51:39.091629 1650 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc89742e6ce64b56a569e695b8459dcd [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:39.092783 1660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:51:39.097779 1660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:51:39.106247 1660 catalog_manager.cc:1347] Generated new cluster ID: 6efa4a94bb4d4aa09d88eb6af93e9ad6
I20260808 03:51:39.106362 1660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:51:39.118698 1660 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:51:39.119079 1660 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:51:39.125926 1660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc89742e6ce64b56a569e695b8459dcd: Generated new TSK 0
I20260808 03:51:39.126720 1660 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:51:39.134493 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:42633
--builtin_ntp_servers=127.26.146.148:46591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 03:51:39.469942 1677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:39.470234 1677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:39.470374 1677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:39.477463 1677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:39.477568 1677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:39.477720 1677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:51:39.486443 1677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:46591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:42633
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:39.487737 1677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:39.489565 1677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:39.502296 1684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:39.502264 1687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:39.502327 1685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:39.503085 1677 server_base.cc:1034] running on GCE node
I20260808 03:51:39.503621 1677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:39.504570 1677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:39.505777 1677 hybrid_clock.cc:648] HybridClock initialized: now 1786161099505735 us; error 32 us; skew 500 ppm
I20260808 03:51:39.506071 1677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:51:39.508270 1677 webserver.cc:460] Webserver started at http://127.26.146.129:44609/ using document root <none> and password file <none>
I20260808 03:51:39.508927 1677 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:39.509029 1677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:39.509359 1677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:39.512204 1677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "77eedd41398f426395f3897bdac5fb07"
format_stamp: "Formatted at 2026-08-08 03:51:39 on dist-test-slave-jlzn"
server_key: "1db1a81f228cf070dd93d44124e65091"
server_key_iv: "bcb415147c1e28a2face7f8371a3863a"
server_key_version: "encryptionkey@0"
I20260808 03:51:39.512970 1677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "77eedd41398f426395f3897bdac5fb07"
format_stamp: "Formatted at 2026-08-08 03:51:39 on dist-test-slave-jlzn"
server_key: "1db1a81f228cf070dd93d44124e65091"
server_key_iv: "bcb415147c1e28a2face7f8371a3863a"
server_key_version: "encryptionkey@0"
I20260808 03:51:39.517973 1677 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:51:39.521279 1695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:39.522512 1677 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 03:51:39.522670 1677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "77eedd41398f426395f3897bdac5fb07"
format_stamp: "Formatted at 2026-08-08 03:51:39 on dist-test-slave-jlzn"
server_key: "1db1a81f228cf070dd93d44124e65091"
server_key_iv: "bcb415147c1e28a2face7f8371a3863a"
server_key_version: "encryptionkey@0"
I20260808 03:51:39.522878 1677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:39.528494 1677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:39.529264 1677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:39.529510 1677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:39.530390 1677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:39.531956 1677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:39.532051 1677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:39.532158 1677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:39.532235 1677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:39.560887 1677 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:46607
I20260808 03:51:39.560943 1858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:46607 every 8 connection(s)
I20260808 03:51:39.562539 1677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:51:39.570245 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:39.565183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:47983 (local address 127.26.146.190:42633)
0808 03:51:39.565379 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:39.565384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:39.566664 (+ 1280us) server_negotiation.cc:407] Connection header received
0808 03:51:39.568391 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:39.568395 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:39.568449 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:39.568537 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:39.569734 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:39.569738 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:39.569741 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:51:39.569753 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:39.569777 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:51:39.569786 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:39.569875 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:39.570003 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 03:51:39.570049 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 03:51:39.570870 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 1677
I20260808 03:51:39.571028 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:51:39.571170 1864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:39.565616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:42633 (local address 127.26.146.129:47983)
0808 03:51:39.566510 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:39.566550 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:39.568133 (+ 1583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:39.568720 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:39.568728 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:39.568819 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:39.569481 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:51:39.569483 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:39.569522 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:51:39.569606 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:39.569855 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:39.569861 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:39.569866 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 03:51:39.570079 (+ 213us) client_negotiation.cc:240] Negotiation successful
0808 03:51:39.570264 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":264,"threads_started":1}
I20260808 03:51:39.571862 27210 external_mini_cluster.cc:1428] Setting key 379b823508a6da5af7b9fe6b0ecc7abb
I20260808 03:51:39.572599 1860 heartbeater.cc:347] Connected to a master server at 127.26.146.190:42633
I20260808 03:51:39.572911 1860 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:39.573501 1860 heartbeater.cc:511] Master 127.26.146.190:42633 requested a full tablet report, sending...
I20260808 03:51:39.574759 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:42633
--builtin_ntp_servers=127.26.146.148:46591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 03:51:39.575400 1596 ts_manager.cc:194] Registered new tserver with Master: 77eedd41398f426395f3897bdac5fb07 (127.26.146.129:46607)
W20260808 03:51:39.896466 1866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:39.896832 1866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:39.896965 1866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:39.904089 1866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:39.904191 1866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:39.904330 1866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:51:39.913269 1866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:46591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:42633
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:39.914597 1866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:39.916455 1866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:39.929564 1873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:39.929610 1875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:39.929646 1877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:39.930473 1866 server_base.cc:1034] running on GCE node
I20260808 03:51:39.931012 1866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:39.932020 1866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:39.933230 1866 hybrid_clock.cc:648] HybridClock initialized: now 1786161099933170 us; error 50 us; skew 500 ppm
I20260808 03:51:39.933527 1866 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:51:39.935786 1866 webserver.cc:460] Webserver started at http://127.26.146.130:35157/ using document root <none> and password file <none>
I20260808 03:51:39.936478 1866 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:39.936573 1866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:39.936868 1866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:39.939626 1866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "7dd6803691404ba9802ccf7e6b0b42d3"
format_stamp: "Formatted at 2026-08-08 03:51:39 on dist-test-slave-jlzn"
server_key: "dc806cb2aaa5643dca3a72e42f7295e2"
server_key_iv: "ba0bdf01c26d873c69232d9f4ccfd9ba"
server_key_version: "encryptionkey@0"
I20260808 03:51:39.940425 1866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "7dd6803691404ba9802ccf7e6b0b42d3"
format_stamp: "Formatted at 2026-08-08 03:51:39 on dist-test-slave-jlzn"
server_key: "dc806cb2aaa5643dca3a72e42f7295e2"
server_key_iv: "ba0bdf01c26d873c69232d9f4ccfd9ba"
server_key_version: "encryptionkey@0"
I20260808 03:51:39.945312 1866 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 03:51:39.948690 1883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:39.949916 1866 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260808 03:51:39.950091 1866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "7dd6803691404ba9802ccf7e6b0b42d3"
format_stamp: "Formatted at 2026-08-08 03:51:39 on dist-test-slave-jlzn"
server_key: "dc806cb2aaa5643dca3a72e42f7295e2"
server_key_iv: "ba0bdf01c26d873c69232d9f4ccfd9ba"
server_key_version: "encryptionkey@0"
I20260808 03:51:39.950289 1866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:39.955789 1866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:39.956610 1866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:39.956853 1866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:39.957705 1866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:39.959025 1866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:39.959131 1866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:39.959240 1866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:39.959311 1866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:39.989284 1866 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:45283
I20260808 03:51:39.989337 2041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:45283 every 8 connection(s)
I20260808 03:51:39.990694 1866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:51:39.995817 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 1866
I20260808 03:51:39.995983 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:51:39.996407 27210 external_mini_cluster.cc:1428] Setting key f6aa4698808f4e17e01058ce0558bfc8
I20260808 03:51:39.998790 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:39.993223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:32813 (local address 127.26.146.190:42633)
0808 03:51:39.993398 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:39.993402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:39.994811 (+ 1409us) server_negotiation.cc:407] Connection header received
0808 03:51:39.996566 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:39.996572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:39.996645 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:39.996739 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:39.998081 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:39.998088 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:39.998093 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:51:39.998108 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:39.998148 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:51:39.998160 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:39.998272 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:39.998448 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 03:51:39.998515 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 03:51:39.999168 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:42633
--builtin_ntp_servers=127.26.146.148:46591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 03:51:39.999761 2044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:39.993766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:42633 (local address 127.26.146.130:32813)
0808 03:51:39.994661 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:39.994699 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:39.996296 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:39.997008 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:39.997019 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:39.997136 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:39.997787 (+ 651us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:51:39.997790 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:39.997828 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:51:39.997915 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:39.998248 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:39.998259 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:39.998267 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 03:51:39.998519 (+ 252us) client_negotiation.cc:240] Negotiation successful
0808 03:51:39.998785 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":262,"threads_started":1}
I20260808 03:51:40.001149 2042 heartbeater.cc:347] Connected to a master server at 127.26.146.190:42633
I20260808 03:51:40.001415 2042 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:40.001943 2042 heartbeater.cc:511] Master 127.26.146.190:42633 requested a full tablet report, sending...
I20260808 03:51:40.003266 1596 ts_manager.cc:194] Registered new tserver with Master: 7dd6803691404ba9802ccf7e6b0b42d3 (127.26.146.130:45283)
W20260808 03:51:40.326344 2045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:40.326625 2045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:40.326736 2045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:40.333877 2045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:40.333984 2045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:40.334141 2045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:51:40.342911 2045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:46591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:42633
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:40.344234 2045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:40.345979 2045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:40.358379 2052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:40.358326 2054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:40.358296 2056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:40.359162 2045 server_base.cc:1034] running on GCE node
I20260808 03:51:40.359714 2045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:40.360601 2045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:40.361802 2045 hybrid_clock.cc:648] HybridClock initialized: now 1786161100361758 us; error 32 us; skew 500 ppm
I20260808 03:51:40.362089 2045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:51:40.364231 2045 webserver.cc:460] Webserver started at http://127.26.146.131:42501/ using document root <none> and password file <none>
I20260808 03:51:40.364871 2045 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:40.364967 2045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:40.365283 2045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:40.368090 2045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "42a6558523d2420ab6e30d908944498b"
format_stamp: "Formatted at 2026-08-08 03:51:40 on dist-test-slave-jlzn"
server_key: "64116fa55652a313ad6c578fb9bc148f"
server_key_iv: "137b5c424868671123d34c1a152b8d36"
server_key_version: "encryptionkey@0"
I20260808 03:51:40.368875 2045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "42a6558523d2420ab6e30d908944498b"
format_stamp: "Formatted at 2026-08-08 03:51:40 on dist-test-slave-jlzn"
server_key: "64116fa55652a313ad6c578fb9bc148f"
server_key_iv: "137b5c424868671123d34c1a152b8d36"
server_key_version: "encryptionkey@0"
I20260808 03:51:40.373714 2045 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 03:51:40.376879 2069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:40.378053 2045 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 03:51:40.378211 2045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "42a6558523d2420ab6e30d908944498b"
format_stamp: "Formatted at 2026-08-08 03:51:40 on dist-test-slave-jlzn"
server_key: "64116fa55652a313ad6c578fb9bc148f"
server_key_iv: "137b5c424868671123d34c1a152b8d36"
server_key_version: "encryptionkey@0"
I20260808 03:51:40.378399 2045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:40.383725 2045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:40.384472 2045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:40.384714 2045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:40.385572 2045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:40.386981 2045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:40.387076 2045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:40.387192 2045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:40.387262 2045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:40.416821 2045 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:46671
I20260808 03:51:40.416870 2239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:46671 every 8 connection(s)
I20260808 03:51:40.418134 2045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:51:40.420029 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 2045
I20260808 03:51:40.420193 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:51:40.420665 27210 external_mini_cluster.cc:1428] Setting key 4e3b458f7c78893987467da593963ea5
I20260808 03:51:40.426326 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:40.420756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:42063 (local address 127.26.146.190:42633)
0808 03:51:40.420978 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:40.420984 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:40.422562 (+ 1578us) server_negotiation.cc:407] Connection header received
0808 03:51:40.424457 (+ 1895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:40.424461 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:40.424513 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:40.424595 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:40.425747 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:40.425751 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:40.425755 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:51:40.425770 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:40.425802 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:51:40.425816 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:40.425928 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:40.426066 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 03:51:40.426116 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 03:51:40.427213 2242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:40.421344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:42633 (local address 127.26.146.131:42063)
0808 03:51:40.422403 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:40.422448 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:40.424226 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:40.424796 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:40.424804 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:40.424891 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:40.425475 (+ 584us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:51:40.425477 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:40.425512 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:51:40.425597 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:40.425904 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:40.425912 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:40.425918 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 03:51:40.426126 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 03:51:40.426317 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":324,"threads_started":1}
I20260808 03:51:40.428409 2240 heartbeater.cc:347] Connected to a master server at 127.26.146.190:42633
I20260808 03:51:40.428655 2240 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:40.429179 2240 heartbeater.cc:511] Master 127.26.146.190:42633 requested a full tablet report, sending...
I20260808 03:51:40.430550 1596 ts_manager.cc:194] Registered new tserver with Master: 42a6558523d2420ab6e30d908944498b (127.26.146.131:46671)
I20260808 03:51:40.435542 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:51:40.442476 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:40.440011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39822 (local address 127.26.146.190:42633)
0808 03:51:40.440165 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:40.440169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:40.440661 (+ 492us) server_negotiation.cc:407] Connection header received
0808 03:51:40.440838 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:40.440842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:40.440893 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:40.440970 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:40.441862 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:40.441868 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:40.441873 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:51:40.441888 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:40.441922 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:51:40.441932 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:40.442034 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:40.442146 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 03:51:40.442212 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 03:51:40.446293 1596 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39822:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:51:40.452967 1596 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:51:40.482561 2256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:40.464546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:46607 (local address 127.0.0.1:60332)
0808 03:51:40.477671 (+ 13125us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:40.477709 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:40.478402 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:40.480172 (+ 1770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:40.480181 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:40.480284 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:40.481157 (+ 873us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:51:40.481161 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:40.481201 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:51:40.481292 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:40.481706 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:40.481718 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:40.481726 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 03:51:40.481928 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 03:51:40.482029 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12874,"thread_start_us":12730,"threads_started":1}
I20260808 03:51:40.482717 2258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:40.464686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60332 (local address 127.26.146.129:46607)
0808 03:51:40.476926 (+ 12240us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:40.476937 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:40.477833 (+ 896us) server_negotiation.cc:407] Connection header received
0808 03:51:40.478607 (+ 774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:40.478619 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:40.478888 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:40.479015 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:40.481479 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:40.481488 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:40.481502 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:51:40.481528 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:40.481589 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:51:40.481611 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:40.481758 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:40.481909 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 03:51:40.482014 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12044,"thread_start_us":12248,"threads_started":1}
I20260808 03:51:40.483161 2257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:40.464600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:45283 (local address 127.0.0.1:33286)
0808 03:51:40.478493 (+ 13893us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:40.478510 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:40.478651 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:40.480378 (+ 1727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:40.480383 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:40.480414 (+ 31us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:40.481449 (+ 1035us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:51:40.481453 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:40.481469 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:51:40.481513 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:40.482005 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:40.482013 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:40.482018 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 03:51:40.482420 (+ 402us) client_negotiation.cc:240] Negotiation successful
0808 03:51:40.482481 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13742,"thread_start_us":14131,"threads_started":1}
I20260808 03:51:40.483736 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:40.464958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33286 (local address 127.26.146.130:45283)
0808 03:51:40.478281 (+ 13323us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:40.478289 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:40.479963 (+ 1674us) server_negotiation.cc:407] Connection header received
0808 03:51:40.480030 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:40.480037 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:40.480222 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:40.480349 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:40.481764 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:40.481771 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:40.481783 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:51:40.481808 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:40.481861 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:51:40.481881 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:40.483215 (+ 1334us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:40.483288 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 03:51:40.483381 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13138,"thread_start_us":13001,"threads_started":1}
I20260808 03:51:40.484300 2262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:40.478394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:46671 (local address 127.0.0.1:47558)
0808 03:51:40.479288 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:40.479305 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:40.479393 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:40.481721 (+ 2328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:40.481725 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:40.481747 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:40.482419 (+ 672us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:51:40.482422 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:40.482441 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:51:40.482480 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:40.483734 (+ 1254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:40.483739 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:40.483743 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 03:51:40.483895 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 03:51:40.483943 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"thread_start_us":643,"threads_started":1}
I20260808 03:51:40.484341 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:40.478442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47558 (local address 127.26.146.131:46671)
0808 03:51:40.481226 (+ 2784us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:40.481240 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:40.481266 (+ 26us) server_negotiation.cc:407] Connection header received
0808 03:51:40.481349 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:40.481357 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:40.481548 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:40.482165 (+ 617us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:40.483476 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:40.483483 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:40.483495 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:51:40.483510 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:40.483564 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:51:40.483581 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:40.483821 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:40.483908 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 03:51:40.483994 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2639,"thread_start_us":2485,"threads_started":1}
I20260808 03:51:40.487324 1764 tablet_service.cc:1467] Processing CreateTablet for tablet 5bb908d098c1486eb41b3a65924e6280 (DEFAULT_TABLE table=test-table [id=22c5b8c5c7ce4f17a539b2192444094b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:40.487331 1958 tablet_service.cc:1467] Processing CreateTablet for tablet 5bb908d098c1486eb41b3a65924e6280 (DEFAULT_TABLE table=test-table [id=22c5b8c5c7ce4f17a539b2192444094b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:40.487705 2146 tablet_service.cc:1467] Processing CreateTablet for tablet 5bb908d098c1486eb41b3a65924e6280 (DEFAULT_TABLE table=test-table [id=22c5b8c5c7ce4f17a539b2192444094b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:40.488581 1764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bb908d098c1486eb41b3a65924e6280. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:40.488802 1958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bb908d098c1486eb41b3a65924e6280. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:40.488930 2146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bb908d098c1486eb41b3a65924e6280. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:40.500104 2264 tablet_bootstrap.cc:492] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07: Bootstrap starting.
I20260808 03:51:40.502803 2265 tablet_bootstrap.cc:492] T 5bb908d098c1486eb41b3a65924e6280 P 7dd6803691404ba9802ccf7e6b0b42d3: Bootstrap starting.
I20260808 03:51:40.504273 2268 tablet_bootstrap.cc:492] T 5bb908d098c1486eb41b3a65924e6280 P 42a6558523d2420ab6e30d908944498b: Bootstrap starting.
I20260808 03:51:40.504503 2264 tablet_bootstrap.cc:654] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:40.505956 2264 log.cc:824] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:40.506361 2265 tablet_bootstrap.cc:654] T 5bb908d098c1486eb41b3a65924e6280 P 7dd6803691404ba9802ccf7e6b0b42d3: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:40.507153 2268 tablet_bootstrap.cc:654] T 5bb908d098c1486eb41b3a65924e6280 P 42a6558523d2420ab6e30d908944498b: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:40.507393 2265 log.cc:824] T 5bb908d098c1486eb41b3a65924e6280 P 7dd6803691404ba9802ccf7e6b0b42d3: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:40.508234 2268 log.cc:824] T 5bb908d098c1486eb41b3a65924e6280 P 42a6558523d2420ab6e30d908944498b: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:40.509986 2264 tablet_bootstrap.cc:492] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07: No bootstrap required, opened a new log
I20260808 03:51:40.510247 2265 tablet_bootstrap.cc:492] T 5bb908d098c1486eb41b3a65924e6280 P 7dd6803691404ba9802ccf7e6b0b42d3: No bootstrap required, opened a new log
I20260808 03:51:40.510281 2264 ts_tablet_manager.cc:1397] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260808 03:51:40.510494 2265 ts_tablet_manager.cc:1397] T 5bb908d098c1486eb41b3a65924e6280 P 7dd6803691404ba9802ccf7e6b0b42d3: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260808 03:51:40.510929 2268 tablet_bootstrap.cc:492] T 5bb908d098c1486eb41b3a65924e6280 P 42a6558523d2420ab6e30d908944498b: No bootstrap required, opened a new log
I20260808 03:51:40.511159 2268 ts_tablet_manager.cc:1397] T 5bb908d098c1486eb41b3a65924e6280 P 42a6558523d2420ab6e30d908944498b: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.002s
I20260808 03:51:40.518313 2268 raft_consensus.cc:348] T 5bb908d098c1486eb41b3a65924e6280 P 42a6558523d2420ab6e30d908944498b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77eedd41398f426395f3897bdac5fb07" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 46607 } } peers { permanent_uuid: "7dd6803691404ba9802ccf7e6b0b42d3" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45283 } } peers { permanent_uuid: "42a6558523d2420ab6e30d908944498b" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 46671 } }
I20260808 03:51:40.518657 2268 raft_consensus.cc:374] T 5bb908d098c1486eb41b3a65924e6280 P 42a6558523d2420ab6e30d908944498b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:40.518784 2268 raft_consensus.cc:729] T 5bb908d098c1486eb41b3a65924e6280 P 42a6558523d2420ab6e30d908944498b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a6558523d2420ab6e30d908944498b, State: Initialized, Role: FOLLOWER
I20260808 03:51:40.519356 2268 consensus_queue.cc:260] T 5bb908d098c1486eb41b3a65924e6280 P 42a6558523d2420ab6e30d908944498b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77eedd41398f426395f3897bdac5fb07" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 46607 } } peers { permanent_uuid: "7dd6803691404ba9802ccf7e6b0b42d3" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45283 } } peers { permanent_uuid: "42a6558523d2420ab6e30d908944498b" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 46671 } }
I20260808 03:51:40.520435 2240 heartbeater.cc:503] Master 127.26.146.190:42633 was elected leader, sending a full tablet report...
I20260808 03:51:40.520496 2265 raft_consensus.cc:348] T 5bb908d098c1486eb41b3a65924e6280 P 7dd6803691404ba9802ccf7e6b0b42d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77eedd41398f426395f3897bdac5fb07" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 46607 } } peers { permanent_uuid: "7dd6803691404ba9802ccf7e6b0b42d3" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45283 } } peers { permanent_uuid: "42a6558523d2420ab6e30d908944498b" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 46671 } }
I20260808 03:51:40.520917 2265 raft_consensus.cc:374] T 5bb908d098c1486eb41b3a65924e6280 P 7dd6803691404ba9802ccf7e6b0b42d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:40.520987 2268 ts_tablet_manager.cc:1428] T 5bb908d098c1486eb41b3a65924e6280 P 42a6558523d2420ab6e30d908944498b: Time spent starting tablet: real 0.010s user 0.008s sys 0.002s
I20260808 03:51:40.521063 2265 raft_consensus.cc:729] T 5bb908d098c1486eb41b3a65924e6280 P 7dd6803691404ba9802ccf7e6b0b42d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd6803691404ba9802ccf7e6b0b42d3, State: Initialized, Role: FOLLOWER
I20260808 03:51:40.521245 2264 raft_consensus.cc:348] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77eedd41398f426395f3897bdac5fb07" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 46607 } } peers { permanent_uuid: "7dd6803691404ba9802ccf7e6b0b42d3" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45283 } } peers { permanent_uuid: "42a6558523d2420ab6e30d908944498b" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 46671 } }
I20260808 03:51:40.521651 2264 raft_consensus.cc:374] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:40.521745 2264 raft_consensus.cc:729] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77eedd41398f426395f3897bdac5fb07, State: Initialized, Role: FOLLOWER
I20260808 03:51:40.521725 2265 consensus_queue.cc:260] T 5bb908d098c1486eb41b3a65924e6280 P 7dd6803691404ba9802ccf7e6b0b42d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77eedd41398f426395f3897bdac5fb07" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 46607 } } peers { permanent_uuid: "7dd6803691404ba9802ccf7e6b0b42d3" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45283 } } peers { permanent_uuid: "42a6558523d2420ab6e30d908944498b" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 46671 } }
I20260808 03:51:40.522328 2264 consensus_queue.cc:260] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77eedd41398f426395f3897bdac5fb07" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 46607 } } peers { permanent_uuid: "7dd6803691404ba9802ccf7e6b0b42d3" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45283 } } peers { permanent_uuid: "42a6558523d2420ab6e30d908944498b" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 46671 } }
I20260808 03:51:40.522781 2042 heartbeater.cc:503] Master 127.26.146.190:42633 was elected leader, sending a full tablet report...
I20260808 03:51:40.523406 2265 ts_tablet_manager.cc:1428] T 5bb908d098c1486eb41b3a65924e6280 P 7dd6803691404ba9802ccf7e6b0b42d3: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260808 03:51:40.524569 1860 heartbeater.cc:503] Master 127.26.146.190:42633 was elected leader, sending a full tablet report...
I20260808 03:51:40.525125 2264 ts_tablet_manager.cc:1428] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
W20260808 03:51:40.564659 1863 tablet.cc:2369] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:51:40.643342 2276 raft_consensus.cc:482] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:51:40.643699 2276 raft_consensus.cc:504] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77eedd41398f426395f3897bdac5fb07" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 46607 } } peers { permanent_uuid: "7dd6803691404ba9802ccf7e6b0b42d3" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45283 } } peers { permanent_uuid: "42a6558523d2420ab6e30d908944498b" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 46671 } }
I20260808 03:51:40.645493 2276 leader_election.cc:288] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd6803691404ba9802ccf7e6b0b42d3 (127.26.146.130:45283), 42a6558523d2420ab6e30d908944498b (127.26.146.131:46671)
I20260808 03:51:40.648752 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:40.645957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:41945 (local address 127.26.146.130:45283)
0808 03:51:40.646416 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:40.646422 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:40.646728 (+ 306us) server_negotiation.cc:407] Connection header received
0808 03:51:40.646904 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:40.646910 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:40.646980 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:40.647082 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:40.648100 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:40.648106 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:40.648110 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:51:40.648146 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:40.648186 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:51:40.648203 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:40.648302 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:40.648413 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 03:51:40.648476 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321}
I20260808 03:51:40.648809 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:40.646201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:59289 (local address 127.26.146.131:46671)
0808 03:51:40.646512 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:40.646518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:40.647009 (+ 491us) server_negotiation.cc:407] Connection header received
0808 03:51:40.647133 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:40.647138 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:40.647197 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:40.647318 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:40.648230 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:40.648234 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:40.648237 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:51:40.648254 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:40.648294 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:51:40.648309 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:40.648430 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:40.648516 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 03:51:40.648571 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191}
I20260808 03:51:40.649003 2277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:40.645717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:45283 (local address 127.26.146.129:41945)
0808 03:51:40.646588 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:40.646616 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:40.646771 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:40.647126 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:40.647131 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:40.647155 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:40.647884 (+ 729us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:51:40.647888 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:40.647907 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:51:40.647958 (+ 51us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:40.648291 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:40.648296 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:40.648300 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 03:51:40.648444 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 03:51:40.648500 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":483,"threads_started":1}
I20260808 03:51:40.649094 2278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:40.645940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:46671 (local address 127.26.146.129:59289)
0808 03:51:40.646894 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:40.646911 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:40.647023 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:40.647347 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:40.647350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:40.647371 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:40.648016 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:51:40.648020 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:40.648037 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:51:40.648081 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:40.648406 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:40.648410 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:40.648415 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 03:51:40.648567 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 03:51:40.648616 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"thread_start_us":621,"threads_started":1}
I20260808 03:51:40.650169 1985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bb908d098c1486eb41b3a65924e6280" candidate_uuid: "77eedd41398f426395f3897bdac5fb07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd6803691404ba9802ccf7e6b0b42d3" is_pre_election: true
I20260808 03:51:40.650170 2177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bb908d098c1486eb41b3a65924e6280" candidate_uuid: "77eedd41398f426395f3897bdac5fb07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42a6558523d2420ab6e30d908944498b" is_pre_election: true
I20260808 03:51:40.650734 2177 raft_consensus.cc:2454] T 5bb908d098c1486eb41b3a65924e6280 P 42a6558523d2420ab6e30d908944498b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77eedd41398f426395f3897bdac5fb07 in term 0.
I20260808 03:51:40.650731 1985 raft_consensus.cc:2454] T 5bb908d098c1486eb41b3a65924e6280 P 7dd6803691404ba9802ccf7e6b0b42d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77eedd41398f426395f3897bdac5fb07 in term 0.
I20260808 03:51:40.651414 1696 leader_election.cc:302] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42a6558523d2420ab6e30d908944498b, 77eedd41398f426395f3897bdac5fb07; no voters:
I20260808 03:51:40.651855 2276 raft_consensus.cc:2781] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:51:40.651998 2276 raft_consensus.cc:482] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:51:40.652093 2276 raft_consensus.cc:3037] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:40.654330 2276 raft_consensus.cc:504] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77eedd41398f426395f3897bdac5fb07" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 46607 } } peers { permanent_uuid: "7dd6803691404ba9802ccf7e6b0b42d3" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45283 } } peers { permanent_uuid: "42a6558523d2420ab6e30d908944498b" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 46671 } }
I20260808 03:51:40.654929 2276 leader_election.cc:288] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07 [CANDIDATE]: Term 1 election: Requested vote from peers 7dd6803691404ba9802ccf7e6b0b42d3 (127.26.146.130:45283), 42a6558523d2420ab6e30d908944498b (127.26.146.131:46671)
I20260808 03:51:40.655308 1985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bb908d098c1486eb41b3a65924e6280" candidate_uuid: "77eedd41398f426395f3897bdac5fb07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd6803691404ba9802ccf7e6b0b42d3"
I20260808 03:51:40.655340 2177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bb908d098c1486eb41b3a65924e6280" candidate_uuid: "77eedd41398f426395f3897bdac5fb07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42a6558523d2420ab6e30d908944498b"
I20260808 03:51:40.655470 1985 raft_consensus.cc:3037] T 5bb908d098c1486eb41b3a65924e6280 P 7dd6803691404ba9802ccf7e6b0b42d3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:40.655515 2177 raft_consensus.cc:3037] T 5bb908d098c1486eb41b3a65924e6280 P 42a6558523d2420ab6e30d908944498b [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:40.657625 1985 raft_consensus.cc:2454] T 5bb908d098c1486eb41b3a65924e6280 P 7dd6803691404ba9802ccf7e6b0b42d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77eedd41398f426395f3897bdac5fb07 in term 1.
I20260808 03:51:40.657694 2177 raft_consensus.cc:2454] T 5bb908d098c1486eb41b3a65924e6280 P 42a6558523d2420ab6e30d908944498b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77eedd41398f426395f3897bdac5fb07 in term 1.
I20260808 03:51:40.657964 1699 leader_election.cc:302] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77eedd41398f426395f3897bdac5fb07, 7dd6803691404ba9802ccf7e6b0b42d3; no voters:
I20260808 03:51:40.658258 2276 raft_consensus.cc:2781] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:40.658677 2276 raft_consensus.cc:686] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07 [term 1 LEADER]: Becoming Leader. State: Replica: 77eedd41398f426395f3897bdac5fb07, State: Running, Role: LEADER
I20260808 03:51:40.659111 2276 consensus_queue.cc:237] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77eedd41398f426395f3897bdac5fb07" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 46607 } } peers { permanent_uuid: "7dd6803691404ba9802ccf7e6b0b42d3" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45283 } } peers { permanent_uuid: "42a6558523d2420ab6e30d908944498b" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 46671 } }
I20260808 03:51:40.663287 1594 catalog_manager.cc:5495] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 77eedd41398f426395f3897bdac5fb07 (127.26.146.129). New cstate: current_term: 1 leader_uuid: "77eedd41398f426395f3897bdac5fb07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77eedd41398f426395f3897bdac5fb07" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 46607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dd6803691404ba9802ccf7e6b0b42d3" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42a6558523d2420ab6e30d908944498b" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 46671 } health_report { overall_health: UNKNOWN } } }
W20260808 03:51:40.669653 2241 tablet.cc:2369] T 5bb908d098c1486eb41b3a65924e6280 P 42a6558523d2420ab6e30d908944498b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:51:40.675379 2258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:40.673592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60344 (local address 127.26.146.129:46607)
0808 03:51:40.673780 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:40.673785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:40.673802 (+ 17us) server_negotiation.cc:407] Connection header received
0808 03:51:40.673991 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:40.673995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:40.674067 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:40.674153 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:40.674856 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:40.674859 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:40.674862 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:51:40.674876 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:40.674907 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:51:40.674918 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:40.675004 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:40.675063 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 03:51:40.675105 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 03:51:40.683199 2177 raft_consensus.cc:1261] T 5bb908d098c1486eb41b3a65924e6280 P 42a6558523d2420ab6e30d908944498b [term 1 FOLLOWER]: Refusing update from remote peer 77eedd41398f426395f3897bdac5fb07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:51:40.683200 1985 raft_consensus.cc:1261] T 5bb908d098c1486eb41b3a65924e6280 P 7dd6803691404ba9802ccf7e6b0b42d3 [term 1 FOLLOWER]: Refusing update from remote peer 77eedd41398f426395f3897bdac5fb07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:51:40.684263 2281 consensus_queue.cc:1035] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd6803691404ba9802ccf7e6b0b42d3" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:51:40.684643 2276 consensus_queue.cc:1035] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42a6558523d2420ab6e30d908944498b" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 46671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:51:40.694481 2286 mvcc.cc:205] Tried to move back new op lower bound from 7316115868388450304 to 7316115868303273984. Current Snapshot: MvccSnapshot[applied={T|T < 7316115868388450304}]
I20260808 03:51:40.695868 2287 mvcc.cc:205] Tried to move back new op lower bound from 7316115868388450304 to 7316115868303273984. Current Snapshot: MvccSnapshot[applied={T|T < 7316115868388450304}]
I20260808 03:51:40.714867 1594 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:51:40.715184 1594 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:51:40.719018 1594 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc89742e6ce64b56a569e695b8459dcd: Sending DeleteTablet for 3 replicas of tablet 5bb908d098c1486eb41b3a65924e6280
I20260808 03:51:40.720274 1764 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bb908d098c1486eb41b3a65924e6280 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:40 UTC) from {username='slave'} at 127.0.0.1:60332
I20260808 03:51:40.720398 1958 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bb908d098c1486eb41b3a65924e6280 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:40 UTC) from {username='slave'} at 127.0.0.1:33286
I20260808 03:51:40.720628 2146 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bb908d098c1486eb41b3a65924e6280 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:40 UTC) from {username='slave'} at 127.0.0.1:47558
I20260808 03:51:40.722028 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 1677
I20260808 03:51:40.732831 2299 tablet_replica.cc:330] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07: stopping tablet replica
I20260808 03:51:40.733321 2299 raft_consensus.cc:2229] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07 [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:51:40.733992 2299 raft_consensus.cc:2258] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:40.738010 2299 ts_tablet_manager.cc:1905] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:40.745450 2299 ts_tablet_manager.cc:1918] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:51:40.745697 2299 log.cc:1196] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-0/wal/wals/5bb908d098c1486eb41b3a65924e6280
I20260808 03:51:40.746346 2299 ts_tablet_manager.cc:1939] T 5bb908d098c1486eb41b3a65924e6280 P 77eedd41398f426395f3897bdac5fb07: Deleting consensus metadata
I20260808 03:51:40.748276 1577 catalog_manager.cc:4841] TS 77eedd41398f426395f3897bdac5fb07 (127.26.146.129:46607): tablet 5bb908d098c1486eb41b3a65924e6280 (table test-table [id=22c5b8c5c7ce4f17a539b2192444094b]) successfully deleted
I20260808 03:51:40.750732 2303 tablet_replica.cc:330] T 5bb908d098c1486eb41b3a65924e6280 P 42a6558523d2420ab6e30d908944498b: stopping tablet replica
I20260808 03:51:40.751123 2303 raft_consensus.cc:2229] T 5bb908d098c1486eb41b3a65924e6280 P 42a6558523d2420ab6e30d908944498b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:51:40.751572 2303 raft_consensus.cc:2258] T 5bb908d098c1486eb41b3a65924e6280 P 42a6558523d2420ab6e30d908944498b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:40.751681 2258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:40.723465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60348 (local address 127.26.146.129:46607)
0808 03:51:40.723645 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:40.723652 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:40.749289 (+ 25637us) server_negotiation.cc:407] Connection header received
0808 03:51:40.749511 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:40.749517 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:40.749588 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:40.749699 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:40.750856 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:40.750862 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:40.750867 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:51:40.750886 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:40.750924 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:51:40.750940 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:40.751035 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:40.751305 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 03:51:40.751368 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 03:51:40.752326 2302 tablet_replica.cc:330] T 5bb908d098c1486eb41b3a65924e6280 P 7dd6803691404ba9802ccf7e6b0b42d3: stopping tablet replica
I20260808 03:51:40.752698 2302 raft_consensus.cc:2229] T 5bb908d098c1486eb41b3a65924e6280 P 7dd6803691404ba9802ccf7e6b0b42d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:51:40.752846 1852 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:40.753188 2302 raft_consensus.cc:2258] T 5bb908d098c1486eb41b3a65924e6280 P 7dd6803691404ba9802ccf7e6b0b42d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:40.755072 2303 ts_tablet_manager.cc:1905] T 5bb908d098c1486eb41b3a65924e6280 P 42a6558523d2420ab6e30d908944498b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:40.755357 2302 ts_tablet_manager.cc:1905] T 5bb908d098c1486eb41b3a65924e6280 P 7dd6803691404ba9802ccf7e6b0b42d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:40.761624 2302 ts_tablet_manager.cc:1918] T 5bb908d098c1486eb41b3a65924e6280 P 7dd6803691404ba9802ccf7e6b0b42d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:51:40.761791 2302 log.cc:1196] T 5bb908d098c1486eb41b3a65924e6280 P 7dd6803691404ba9802ccf7e6b0b42d3: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/5bb908d098c1486eb41b3a65924e6280
I20260808 03:51:40.762209 2302 ts_tablet_manager.cc:1939] T 5bb908d098c1486eb41b3a65924e6280 P 7dd6803691404ba9802ccf7e6b0b42d3: Deleting consensus metadata
I20260808 03:51:40.762777 2303 ts_tablet_manager.cc:1918] T 5bb908d098c1486eb41b3a65924e6280 P 42a6558523d2420ab6e30d908944498b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:51:40.762944 2303 log.cc:1196] T 5bb908d098c1486eb41b3a65924e6280 P 42a6558523d2420ab6e30d908944498b: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/5bb908d098c1486eb41b3a65924e6280
I20260808 03:51:40.763447 2303 ts_tablet_manager.cc:1939] T 5bb908d098c1486eb41b3a65924e6280 P 42a6558523d2420ab6e30d908944498b: Deleting consensus metadata
I20260808 03:51:40.763783 1579 catalog_manager.cc:4841] TS 7dd6803691404ba9802ccf7e6b0b42d3 (127.26.146.130:45283): tablet 5bb908d098c1486eb41b3a65924e6280 (table test-table [id=22c5b8c5c7ce4f17a539b2192444094b]) successfully deleted
I20260808 03:51:40.765172 1577 catalog_manager.cc:4841] TS 42a6558523d2420ab6e30d908944498b (127.26.146.131:46671): tablet 5bb908d098c1486eb41b3a65924e6280 (table test-table [id=22c5b8c5c7ce4f17a539b2192444094b]) successfully deleted
I20260808 03:51:40.851312 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 1677
I20260808 03:51:40.881971 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 1866
I20260808 03:51:40.884702 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:40.882745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33288 (local address 127.26.146.130:45283)
0808 03:51:40.882927 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:40.882931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:40.882990 (+ 59us) server_negotiation.cc:407] Connection header received
0808 03:51:40.883155 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:40.883160 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:40.883245 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:40.883326 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:40.884148 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:40.884152 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:40.884155 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:51:40.884170 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:40.884204 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:51:40.884216 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:40.884316 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:40.884381 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 03:51:40.884426 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 03:51:40.885324 2037 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:40.971704 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 1866
I20260808 03:51:40.994869 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 2045
I20260808 03:51:40.997679 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:40.995614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47570 (local address 127.26.146.131:46671)
0808 03:51:40.995789 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:40.995795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:40.995844 (+ 49us) server_negotiation.cc:407] Connection header received
0808 03:51:40.995997 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:40.996001 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:40.996080 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:40.996181 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:40.997060 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:40.997065 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:40.997068 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:51:40.997083 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:40.997126 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:51:40.997139 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:40.997256 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:40.997340 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 03:51:40.997400 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 03:51:40.998262 2233 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:41.083107 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 2045
I20260808 03:51:41.107796 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 1555
I20260808 03:51:41.108960 1638 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:41.213337 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 1555
2026-08-08T03:51:41Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2654 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 03:51:41.260109 27210 mini_oidc.cc:152] Starting JWKS server
I20260808 03:51:41.262951 27210 webserver.cc:460] Webserver started at https://127.0.0.1:43239/ using document root <none> and password file <none>
I20260808 03:51:41.263329 27210 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 03:51:41.264034 27210 webserver.cc:460] Webserver started at http://127.0.0.1:46751/ using document root <none> and password file <none>
2026-08-08T03:51:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:51:41Z Disabled control of system clock
I20260808 03:51:41.295722 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:34097
--webserver_interface=127.26.146.190
--webserver_port=0
--builtin_ntp_servers=127.26.146.148:41275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:34097
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43239/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786161058098901-27210-0/testchainca.cert with env {}
W20260808 03:51:41.618816 2317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:41.619125 2317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:41.619266 2317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:41.626376 2317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:51:41.626494 2317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:41.626565 2317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:41.626649 2317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:51:41.626724 2317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 03:51:41.626811 2317 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 03:51:41.626866 2317 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43239/jwks.json
I20260808 03:51:41.636705 2317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:41275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:34097
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:34097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43239/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786161058098901-27210-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:41.638017 2317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:41.639308 2317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:41.651023 2326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:41.651023 2328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:41.651057 2322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:41.651772 2317 server_base.cc:1034] running on GCE node
I20260808 03:51:41.652586 2317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:41.653993 2317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:41.655207 2317 hybrid_clock.cc:648] HybridClock initialized: now 1786161101655158 us; error 38 us; skew 500 ppm
I20260808 03:51:41.655555 2317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:51:41.657823 2317 webserver.cc:460] Webserver started at http://127.26.146.190:38163/ using document root <none> and password file <none>
I20260808 03:51:41.658506 2317 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:41.658623 2317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:41.659004 2317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:41.662029 2317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "386d211f74af4d7687a235b83fab9c40"
format_stamp: "Formatted at 2026-08-08 03:51:41 on dist-test-slave-jlzn"
server_key: "42ef4e8b600da7379cc41eedf5aed7a9"
server_key_iv: "9dc877a3de4a77249d0c9f59cb4bab30"
server_key_version: "encryptionkey@0"
I20260808 03:51:41.662866 2317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "386d211f74af4d7687a235b83fab9c40"
format_stamp: "Formatted at 2026-08-08 03:51:41 on dist-test-slave-jlzn"
server_key: "42ef4e8b600da7379cc41eedf5aed7a9"
server_key_iv: "9dc877a3de4a77249d0c9f59cb4bab30"
server_key_version: "encryptionkey@0"
I20260808 03:51:41.668139 2317 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 03:51:41.671427 2336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:41.672719 2317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:51:41.672891 2317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "386d211f74af4d7687a235b83fab9c40"
format_stamp: "Formatted at 2026-08-08 03:51:41 on dist-test-slave-jlzn"
server_key: "42ef4e8b600da7379cc41eedf5aed7a9"
server_key_iv: "9dc877a3de4a77249d0c9f59cb4bab30"
server_key_version: "encryptionkey@0"
I20260808 03:51:41.673089 2317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:41.686851 2317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:41.687899 2317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:41.688225 2317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:41.705459 2317 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:34097
I20260808 03:51:41.705479 2411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:34097 every 8 connection(s)
I20260808 03:51:41.706768 2317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:51:41.710392 2412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:41.713279 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 2317
I20260808 03:51:41.713500 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:51:41.714272 27210 external_mini_cluster.cc:1428] Setting key 68c564a14a278d1db6ee34c7df84fd83
I20260808 03:51:41.722172 2412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 386d211f74af4d7687a235b83fab9c40: Bootstrap starting.
I20260808 03:51:41.725538 2412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 386d211f74af4d7687a235b83fab9c40: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:41.726634 2412 log.cc:824] T 00000000000000000000000000000000 P 386d211f74af4d7687a235b83fab9c40: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:41.727494 2416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:41.716225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50546 (local address 127.26.146.190:34097)
0808 03:51:41.717142 (+ 917us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:41.717153 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:41.717201 (+ 48us) server_negotiation.cc:407] Connection header received
0808 03:51:41.718869 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:41.718892 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:41.719365 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:41.719717 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:41.720324 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:41.721594 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:41.722941 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:41.723660 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:41.723865 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:41.725073 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:41.725080 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:41.725094 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:51:41.725610 (+ 516us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:41.725720 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:51:41.725742 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:41.725849 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:41.726081 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 03:51:41.726369 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":320,"threads_started":1}
I20260808 03:51:41.729736 2412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 386d211f74af4d7687a235b83fab9c40: No bootstrap required, opened a new log
I20260808 03:51:41.731230 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:51:41.737357 2412 raft_consensus.cc:348] T 00000000000000000000000000000000 P 386d211f74af4d7687a235b83fab9c40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386d211f74af4d7687a235b83fab9c40" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 34097 } }
I20260808 03:51:41.737700 2412 raft_consensus.cc:374] T 00000000000000000000000000000000 P 386d211f74af4d7687a235b83fab9c40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:41.737787 2412 raft_consensus.cc:729] T 00000000000000000000000000000000 P 386d211f74af4d7687a235b83fab9c40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 386d211f74af4d7687a235b83fab9c40, State: Initialized, Role: FOLLOWER
I20260808 03:51:41.738251 2412 consensus_queue.cc:260] T 00000000000000000000000000000000 P 386d211f74af4d7687a235b83fab9c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386d211f74af4d7687a235b83fab9c40" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 34097 } }
I20260808 03:51:41.738440 2412 raft_consensus.cc:388] T 00000000000000000000000000000000 P 386d211f74af4d7687a235b83fab9c40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:51:41.738538 2412 raft_consensus.cc:482] T 00000000000000000000000000000000 P 386d211f74af4d7687a235b83fab9c40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:51:41.738669 2412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 386d211f74af4d7687a235b83fab9c40 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:41.741022 2412 raft_consensus.cc:504] T 00000000000000000000000000000000 P 386d211f74af4d7687a235b83fab9c40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386d211f74af4d7687a235b83fab9c40" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 34097 } }
I20260808 03:51:41.741480 2412 leader_election.cc:302] T 00000000000000000000000000000000 P 386d211f74af4d7687a235b83fab9c40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 386d211f74af4d7687a235b83fab9c40; no voters:
I20260808 03:51:41.741990 2412 leader_election.cc:288] T 00000000000000000000000000000000 P 386d211f74af4d7687a235b83fab9c40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:51:41.742107 2418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 386d211f74af4d7687a235b83fab9c40 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:41.742779 2418 raft_consensus.cc:686] T 00000000000000000000000000000000 P 386d211f74af4d7687a235b83fab9c40 [term 1 LEADER]: Becoming Leader. State: Replica: 386d211f74af4d7687a235b83fab9c40, State: Running, Role: LEADER
I20260808 03:51:41.743336 2418 consensus_queue.cc:237] T 00000000000000000000000000000000 P 386d211f74af4d7687a235b83fab9c40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386d211f74af4d7687a235b83fab9c40" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 34097 } }
I20260808 03:51:41.744169 2412 sys_catalog.cc:564] T 00000000000000000000000000000000 P 386d211f74af4d7687a235b83fab9c40 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:51:41.746290 2421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 386d211f74af4d7687a235b83fab9c40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 386d211f74af4d7687a235b83fab9c40. Latest consensus state: current_term: 1 leader_uuid: "386d211f74af4d7687a235b83fab9c40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386d211f74af4d7687a235b83fab9c40" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 34097 } } }
I20260808 03:51:41.746282 2420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 386d211f74af4d7687a235b83fab9c40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "386d211f74af4d7687a235b83fab9c40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386d211f74af4d7687a235b83fab9c40" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 34097 } } }
I20260808 03:51:41.746783 2421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 386d211f74af4d7687a235b83fab9c40 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:41.746799 2420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 386d211f74af4d7687a235b83fab9c40 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:41.748400 2428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:51:41.753490 2428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:51:41.762195 2428 catalog_manager.cc:1347] Generated new cluster ID: 7e3c110af20a4524aa1212cfbc13582d
I20260808 03:51:41.762315 2428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:51:41.784222 2428 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:51:41.785605 2428 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:51:41.793304 2428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 386d211f74af4d7687a235b83fab9c40: Generated new TSK 0
I20260808 03:51:41.794102 2428 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 03:51:41.836565 2451 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 03:51:41.836772 2451 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 03:51:41.837984 2416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:41.815530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50552 (local address 127.26.146.190:34097)
0808 03:51:41.815729 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:41.815735 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:41.816220 (+ 485us) server_negotiation.cc:407] Connection header received
0808 03:51:41.816566 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:41.816572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:41.816683 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:41.816839 (+ 156us) server_negotiation.cc:226] Negotiated authn=JWT
0808 03:51:41.817396 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:41.818593 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:41.820130 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:41.820793 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:41.820921 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:41.821465 (+ 544us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 03:51:41.837292 (+ 15827us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43239/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 03:51:41.837669 (+ 377us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50552: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43239/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 03:51:41.838116 2416 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50552: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43239/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 03:51:41.839530 2450 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:51:41.815373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:34097 (local address 127.0.0.1:50552)
0808 03:51:41.816120 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:41.816137 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:41.816283 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:41.816904 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:41.816910 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:41.816921 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0808 03:51:41.817249 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:41.817258 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:41.818765 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:41.818770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:41.819984 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:41.820001 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:41.820978 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:41.820984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:41.821124 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:41.821193 (+ 69us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 03:51:41.837572 (+ 16379us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43239/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 03:51:41.837863 (+ 291us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.146.190:34097: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43239/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":401,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.26.146.190:34097: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43239/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 03:51:41.840920 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 2317
I20260808 03:51:41.842065 2405 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:41.935201 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 2317
2026-08-08T03:51:41Z chronyd exiting
I20260808 03:51:41.990628 27210 test_util.cc:175] -----------------------------------------------
I20260808 03:51:41.990779 27210 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786161058098901-27210-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (735 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 03:51:41.994678 27210 mini_oidc.cc:152] Starting JWKS server
I20260808 03:51:41.996531 27210 webserver.cc:460] Webserver started at https://127.0.0.1:45911/ using document root <none> and password file <none>
I20260808 03:51:41.996752 27210 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 03:51:41.997421 27210 webserver.cc:460] Webserver started at http://127.0.0.1:42929/ using document root <none> and password file <none>
2026-08-08T03:51:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:51:42Z Disabled control of system clock
I20260808 03:51:42.028986 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:44717
--webserver_interface=127.26.146.190
--webserver_port=0
--builtin_ntp_servers=127.26.146.148:39585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:44717
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45911/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786161058098901-27210-0/testchainca.cert with env {}
W20260808 03:51:42.350996 2459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:42.351293 2459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:42.351437 2459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:42.358632 2459 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:51:42.358762 2459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:42.358831 2459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:42.358913 2459 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:51:42.358974 2459 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 03:51:42.359059 2459 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 03:51:42.359114 2459 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45911/jwks.json
I20260808 03:51:42.369782 2459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:39585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:44717
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:44717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45911/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786161058098901-27210-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:42.371090 2459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:42.372452 2459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:42.384070 2475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:42.384128 2470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:42.384161 2466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:42.384930 2459 server_base.cc:1034] running on GCE node
I20260808 03:51:42.385649 2459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:42.386998 2459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:42.388185 2459 hybrid_clock.cc:648] HybridClock initialized: now 1786161102388151 us; error 30 us; skew 500 ppm
I20260808 03:51:42.388489 2459 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:51:42.394522 2459 webserver.cc:460] Webserver started at http://127.26.146.190:36353/ using document root <none> and password file <none>
I20260808 03:51:42.395176 2459 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:42.395270 2459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:42.395545 2459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:42.398475 2459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "9092e4ee8a3449f885ad45bff7c39cb9"
format_stamp: "Formatted at 2026-08-08 03:51:42 on dist-test-slave-jlzn"
server_key: "8dbe4a84365b71f43710156ca70225d0"
server_key_iv: "db4a2a267cdd8f46b8d8449378389b9e"
server_key_version: "encryptionkey@0"
I20260808 03:51:42.399246 2459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "9092e4ee8a3449f885ad45bff7c39cb9"
format_stamp: "Formatted at 2026-08-08 03:51:42 on dist-test-slave-jlzn"
server_key: "8dbe4a84365b71f43710156ca70225d0"
server_key_iv: "db4a2a267cdd8f46b8d8449378389b9e"
server_key_version: "encryptionkey@0"
I20260808 03:51:42.404258 2459 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 03:51:42.407656 2482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:42.409016 2459 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:51:42.409188 2459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "9092e4ee8a3449f885ad45bff7c39cb9"
format_stamp: "Formatted at 2026-08-08 03:51:42 on dist-test-slave-jlzn"
server_key: "8dbe4a84365b71f43710156ca70225d0"
server_key_iv: "db4a2a267cdd8f46b8d8449378389b9e"
server_key_version: "encryptionkey@0"
I20260808 03:51:42.409387 2459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:42.440167 2459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:42.441016 2459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:42.441306 2459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:42.458415 2459 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:44717
I20260808 03:51:42.458433 2553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:44717 every 8 connection(s)
I20260808 03:51:42.459729 2459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:51:42.463356 2554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:42.467348 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 2459
I20260808 03:51:42.467557 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:51:42.468472 27210 external_mini_cluster.cc:1428] Setting key a79460ae1c715bde1d3a3f468d280ffa
I20260808 03:51:42.475020 2554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9092e4ee8a3449f885ad45bff7c39cb9: Bootstrap starting.
I20260808 03:51:42.478559 2554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9092e4ee8a3449f885ad45bff7c39cb9: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:42.479600 2554 log.cc:824] T 00000000000000000000000000000000 P 9092e4ee8a3449f885ad45bff7c39cb9: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:42.482213 2564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:42.470331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42144 (local address 127.26.146.190:44717)
0808 03:51:42.471273 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:42.471283 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:42.471322 (+ 39us) server_negotiation.cc:407] Connection header received
0808 03:51:42.473435 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:42.473459 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:42.473947 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:42.474275 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:42.474844 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:42.476209 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:42.477972 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:42.478653 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:42.478856 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:42.480002 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:42.480010 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:42.480024 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:51:42.480578 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:42.480663 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:51:42.480687 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:42.480793 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:42.481014 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 03:51:42.481326 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":329,"threads_started":1}
I20260808 03:51:42.482547 2554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9092e4ee8a3449f885ad45bff7c39cb9: No bootstrap required, opened a new log
I20260808 03:51:42.485735 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:51:42.490170 2554 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9092e4ee8a3449f885ad45bff7c39cb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9092e4ee8a3449f885ad45bff7c39cb9" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44717 } }
I20260808 03:51:42.490506 2554 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9092e4ee8a3449f885ad45bff7c39cb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:42.490607 2554 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9092e4ee8a3449f885ad45bff7c39cb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9092e4ee8a3449f885ad45bff7c39cb9, State: Initialized, Role: FOLLOWER
I20260808 03:51:42.491119 2554 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9092e4ee8a3449f885ad45bff7c39cb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9092e4ee8a3449f885ad45bff7c39cb9" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44717 } }
I20260808 03:51:42.491343 2554 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9092e4ee8a3449f885ad45bff7c39cb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:51:42.491469 2554 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9092e4ee8a3449f885ad45bff7c39cb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:51:42.491645 2554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9092e4ee8a3449f885ad45bff7c39cb9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:42.493927 2554 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9092e4ee8a3449f885ad45bff7c39cb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9092e4ee8a3449f885ad45bff7c39cb9" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44717 } }
I20260808 03:51:42.494428 2554 leader_election.cc:302] T 00000000000000000000000000000000 P 9092e4ee8a3449f885ad45bff7c39cb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9092e4ee8a3449f885ad45bff7c39cb9; no voters:
I20260808 03:51:42.495064 2554 leader_election.cc:288] T 00000000000000000000000000000000 P 9092e4ee8a3449f885ad45bff7c39cb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:51:42.495153 2566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9092e4ee8a3449f885ad45bff7c39cb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:42.495839 2566 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9092e4ee8a3449f885ad45bff7c39cb9 [term 1 LEADER]: Becoming Leader. State: Replica: 9092e4ee8a3449f885ad45bff7c39cb9, State: Running, Role: LEADER
I20260808 03:51:42.496361 2566 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9092e4ee8a3449f885ad45bff7c39cb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9092e4ee8a3449f885ad45bff7c39cb9" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44717 } }
I20260808 03:51:42.497134 2554 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9092e4ee8a3449f885ad45bff7c39cb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:51:42.498906 2568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9092e4ee8a3449f885ad45bff7c39cb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9092e4ee8a3449f885ad45bff7c39cb9. Latest consensus state: current_term: 1 leader_uuid: "9092e4ee8a3449f885ad45bff7c39cb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9092e4ee8a3449f885ad45bff7c39cb9" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44717 } } }
I20260808 03:51:42.499333 2568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9092e4ee8a3449f885ad45bff7c39cb9 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:42.500574 2576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:51:42.500403 2567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9092e4ee8a3449f885ad45bff7c39cb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9092e4ee8a3449f885ad45bff7c39cb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9092e4ee8a3449f885ad45bff7c39cb9" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44717 } } }
I20260808 03:51:42.500820 2567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9092e4ee8a3449f885ad45bff7c39cb9 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:42.506038 2576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:51:42.515250 2576 catalog_manager.cc:1347] Generated new cluster ID: c8fc8b0a999d4d159232ec7504e25312
I20260808 03:51:42.515363 2576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:51:42.543051 2576 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:51:42.544463 2576 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:51:42.551527 2576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9092e4ee8a3449f885ad45bff7c39cb9: Generated new TSK 0
I20260808 03:51:42.552613 2576 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 03:51:42.594179 2599 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 03:51:42.594295 2599 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 03:51:42.595521 2564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:42.575144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42154 (local address 127.26.146.190:44717)
0808 03:51:42.575346 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:42.575356 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:42.575858 (+ 502us) server_negotiation.cc:407] Connection header received
0808 03:51:42.576109 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:42.576118 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:42.576217 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:42.576376 (+ 159us) server_negotiation.cc:226] Negotiated authn=JWT
0808 03:51:42.576887 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:42.578034 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:42.579902 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:42.580572 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:42.580703 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:42.581121 (+ 418us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 03:51:42.594858 (+ 13737us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45911/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 03:51:42.595230 (+ 372us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42154: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45911/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":62}
W20260808 03:51:42.595556 2598 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:51:42.575144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:44717 (local address 127.0.0.1:42154)
0808 03:51:42.575753 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:42.575772 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:42.575923 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:42.576423 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:42.576427 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:42.576438 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0808 03:51:42.576720 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:42.576729 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:42.578215 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:42.578221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:42.579744 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:42.579759 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:42.580715 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:42.580719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:42.580835 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:42.580866 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 03:51:42.595075 (+ 14209us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45911/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 03:51:42.595277 (+ 202us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.146.190:44717: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45911/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":298,"threads_started":1}
W20260808 03:51:42.595664 2564 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42154: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45911/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260808 03:51:42.596959 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 2459
I20260808 03:51:42.598212 2548 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:42.692492 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 2459
2026-08-08T03:51:42Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (757 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 03:51:42.752436 27210 mini_oidc.cc:152] Starting JWKS server
I20260808 03:51:42.754284 27210 webserver.cc:460] Webserver started at https://127.0.0.1:38505/ using document root <none> and password file <none>
I20260808 03:51:42.754525 27210 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 03:51:42.755105 27210 webserver.cc:460] Webserver started at http://127.0.0.1:45607/ using document root <none> and password file <none>
2026-08-08T03:51:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:51:42Z Disabled control of system clock
I20260808 03:51:42.787526 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:35327
--webserver_interface=127.26.146.190
--webserver_port=0
--builtin_ntp_servers=127.26.146.148:33139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:35327
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38505/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 03:51:43.109707 2609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:43.110000 2609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:43.110137 2609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:43.117389 2609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:51:43.117515 2609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:43.117594 2609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:43.117676 2609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:51:43.117734 2609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 03:51:43.117820 2609 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 03:51:43.117872 2609 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38505/jwks.json
I20260808 03:51:43.127285 2609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:33139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:35327
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:35327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38505/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:43.128615 2609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:43.130424 2609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:43.141955 2616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:43.141975 2617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:43.142624 2619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:43.143095 2609 server_base.cc:1034] running on GCE node
I20260808 03:51:43.143887 2609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:43.145195 2609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:43.146395 2609 hybrid_clock.cc:648] HybridClock initialized: now 1786161103146343 us; error 39 us; skew 500 ppm
I20260808 03:51:43.146719 2609 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:51:43.148972 2609 webserver.cc:460] Webserver started at http://127.26.146.190:34529/ using document root <none> and password file <none>
I20260808 03:51:43.149649 2609 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:43.149747 2609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:43.150072 2609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:43.152974 2609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "54428b4ada6b4bfd9e9517829ede24e9"
format_stamp: "Formatted at 2026-08-08 03:51:43 on dist-test-slave-jlzn"
server_key: "4e9afa41891a8206156dffe43ddda5bb"
server_key_iv: "822953b45bda47e6b4a3b17963216d18"
server_key_version: "encryptionkey@0"
I20260808 03:51:43.153757 2609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "54428b4ada6b4bfd9e9517829ede24e9"
format_stamp: "Formatted at 2026-08-08 03:51:43 on dist-test-slave-jlzn"
server_key: "4e9afa41891a8206156dffe43ddda5bb"
server_key_iv: "822953b45bda47e6b4a3b17963216d18"
server_key_version: "encryptionkey@0"
I20260808 03:51:43.158936 2609 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 03:51:43.162478 2629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:43.163735 2609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 03:51:43.163970 2609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "54428b4ada6b4bfd9e9517829ede24e9"
format_stamp: "Formatted at 2026-08-08 03:51:43 on dist-test-slave-jlzn"
server_key: "4e9afa41891a8206156dffe43ddda5bb"
server_key_iv: "822953b45bda47e6b4a3b17963216d18"
server_key_version: "encryptionkey@0"
I20260808 03:51:43.164206 2609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:43.191565 2609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:43.192463 2609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:43.192765 2609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:43.210721 2609 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:35327
I20260808 03:51:43.210740 2693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:35327 every 8 connection(s)
I20260808 03:51:43.212108 2609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:51:43.214690 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 2609
I20260808 03:51:43.214885 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:51:43.215503 27210 external_mini_cluster.cc:1428] Setting key 64b0d06ba330a82c3f47d5ce17f78f91
I20260808 03:51:43.215910 2694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:43.228186 2694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54428b4ada6b4bfd9e9517829ede24e9: Bootstrap starting.
I20260808 03:51:43.228926 2697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:43.217451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42210 (local address 127.26.146.190:35327)
0808 03:51:43.218391 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:43.218402 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:43.218458 (+ 56us) server_negotiation.cc:407] Connection header received
0808 03:51:43.220634 (+ 2176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:43.220657 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:43.221131 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:43.221504 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:43.222082 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:43.223373 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:43.224718 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:43.225387 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:43.225572 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:43.226694 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:43.226701 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:43.226714 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:51:43.227207 (+ 493us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:43.227283 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:51:43.227304 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:43.227410 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:43.227633 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 03:51:43.227950 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":336,"threads_started":1}
I20260808 03:51:43.231597 2694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54428b4ada6b4bfd9e9517829ede24e9: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:43.232836 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:51:43.232821 2694 log.cc:824] T 00000000000000000000000000000000 P 54428b4ada6b4bfd9e9517829ede24e9: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:43.235579 2694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54428b4ada6b4bfd9e9517829ede24e9: No bootstrap required, opened a new log
I20260808 03:51:43.242846 2694 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54428b4ada6b4bfd9e9517829ede24e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54428b4ada6b4bfd9e9517829ede24e9" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 35327 } }
I20260808 03:51:43.243156 2694 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54428b4ada6b4bfd9e9517829ede24e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:43.243229 2694 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54428b4ada6b4bfd9e9517829ede24e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54428b4ada6b4bfd9e9517829ede24e9, State: Initialized, Role: FOLLOWER
I20260808 03:51:43.243690 2694 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54428b4ada6b4bfd9e9517829ede24e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54428b4ada6b4bfd9e9517829ede24e9" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 35327 } }
I20260808 03:51:43.243913 2694 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54428b4ada6b4bfd9e9517829ede24e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:51:43.244019 2694 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54428b4ada6b4bfd9e9517829ede24e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:51:43.244138 2694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54428b4ada6b4bfd9e9517829ede24e9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:43.246069 2694 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54428b4ada6b4bfd9e9517829ede24e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54428b4ada6b4bfd9e9517829ede24e9" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 35327 } }
I20260808 03:51:43.246481 2694 leader_election.cc:302] T 00000000000000000000000000000000 P 54428b4ada6b4bfd9e9517829ede24e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54428b4ada6b4bfd9e9517829ede24e9; no voters:
I20260808 03:51:43.246996 2694 leader_election.cc:288] T 00000000000000000000000000000000 P 54428b4ada6b4bfd9e9517829ede24e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:51:43.247138 2699 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54428b4ada6b4bfd9e9517829ede24e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:43.247781 2699 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54428b4ada6b4bfd9e9517829ede24e9 [term 1 LEADER]: Becoming Leader. State: Replica: 54428b4ada6b4bfd9e9517829ede24e9, State: Running, Role: LEADER
I20260808 03:51:43.248284 2699 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54428b4ada6b4bfd9e9517829ede24e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54428b4ada6b4bfd9e9517829ede24e9" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 35327 } }
I20260808 03:51:43.248967 2694 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54428b4ada6b4bfd9e9517829ede24e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:51:43.250967 2702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54428b4ada6b4bfd9e9517829ede24e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54428b4ada6b4bfd9e9517829ede24e9. Latest consensus state: current_term: 1 leader_uuid: "54428b4ada6b4bfd9e9517829ede24e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54428b4ada6b4bfd9e9517829ede24e9" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 35327 } } }
I20260808 03:51:43.251013 2700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54428b4ada6b4bfd9e9517829ede24e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54428b4ada6b4bfd9e9517829ede24e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54428b4ada6b4bfd9e9517829ede24e9" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 35327 } } }
I20260808 03:51:43.251340 2702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54428b4ada6b4bfd9e9517829ede24e9 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:43.251410 2700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54428b4ada6b4bfd9e9517829ede24e9 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:43.253093 2708 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:51:43.258141 2708 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:51:43.266737 2708 catalog_manager.cc:1347] Generated new cluster ID: 21f97415cf494ecf9fb411e866a522f2
I20260808 03:51:43.266836 2708 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:51:43.278255 2708 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:51:43.279551 2708 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:51:43.285673 2708 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54428b4ada6b4bfd9e9517829ede24e9: Generated new TSK 0
I20260808 03:51:43.286383 2708 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:51:43.307031 2697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:43.300005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42214 (local address 127.26.146.190:35327)
0808 03:51:43.300182 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:43.300188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:43.300540 (+ 352us) server_negotiation.cc:407] Connection header received
0808 03:51:43.300784 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:43.300788 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:43.300861 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:43.301026 (+ 165us) server_negotiation.cc:226] Negotiated authn=JWT
0808 03:51:43.301513 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:43.302222 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:43.303549 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:43.304046 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:43.304166 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:43.304650 (+ 484us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 03:51:43.306437 (+ 1787us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38505/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 03:51:43.306737 (+ 300us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42214: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38505/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 03:51:43.307224 2732 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:51:43.299908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:35327 (local address 127.0.0.1:42214)
0808 03:51:43.300443 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:43.300458 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:43.300595 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:43.301038 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:43.301043 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:43.301055 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0808 03:51:43.301366 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:43.301376 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:43.302363 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:43.302367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:43.303409 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:43.303420 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:43.304233 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:43.304240 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:43.304375 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:43.304409 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 03:51:43.306632 (+ 2223us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38505/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 03:51:43.306852 (+ 220us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.146.190:35327: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38505/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":300,"threads_started":1}
W20260808 03:51:43.307487 2697 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42214: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38505/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260808 03:51:43.308349 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 2609
I20260808 03:51:43.309345 2688 generic_service.cc:196] Checking for leaks (request via RPC)
E20260808 03:51:43.318981 2734 webserver.cc:533] Webserver: sslize failed:
E20260808 03:51:43.319087 2734 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 03:51:43.399076 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 2609
2026-08-08T03:51:43Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (703 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T03:51:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:51:43Z Disabled control of system clock
I20260808 03:51:43.487653 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:40115
--webserver_interface=127.26.146.190
--webserver_port=0
--builtin_ntp_servers=127.26.146.148:42493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:40115
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786161058098901-27210-0/testchainca.cert with env {}
W20260808 03:51:43.814684 2742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:43.814986 2742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:43.815116 2742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:43.822197 2742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:51:43.822310 2742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:43.822374 2742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:43.822451 2742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:51:43.822521 2742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 03:51:43.822609 2742 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 03:51:43.822664 2742 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 03:51:43.832955 2742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:40115
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:40115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786161058098901-27210-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:43.834283 2742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:43.835561 2742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:43.847137 2752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:43.847470 2750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:43.847470 2749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:43.848256 2742 server_base.cc:1034] running on GCE node
I20260808 03:51:43.848968 2742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:43.850390 2742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:43.851590 2742 hybrid_clock.cc:648] HybridClock initialized: now 1786161103851553 us; error 31 us; skew 500 ppm
I20260808 03:51:43.851934 2742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:51:43.854072 2742 webserver.cc:460] Webserver started at http://127.26.146.190:33929/ using document root <none> and password file <none>
I20260808 03:51:43.854727 2742 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:43.854816 2742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:43.855064 2742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:43.857899 2742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "1a28d12d388847a7a93476f0562a203d"
format_stamp: "Formatted at 2026-08-08 03:51:43 on dist-test-slave-jlzn"
server_key: "e08f4856a959dcf76ad509da14ae637f"
server_key_iv: "5398fbcea8719c547cb962aa3419b521"
server_key_version: "encryptionkey@0"
I20260808 03:51:43.858633 2742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "1a28d12d388847a7a93476f0562a203d"
format_stamp: "Formatted at 2026-08-08 03:51:43 on dist-test-slave-jlzn"
server_key: "e08f4856a959dcf76ad509da14ae637f"
server_key_iv: "5398fbcea8719c547cb962aa3419b521"
server_key_version: "encryptionkey@0"
I20260808 03:51:43.863478 2742 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 03:51:43.866730 2763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:43.868042 2742 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 03:51:43.868224 2742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "1a28d12d388847a7a93476f0562a203d"
format_stamp: "Formatted at 2026-08-08 03:51:43 on dist-test-slave-jlzn"
server_key: "e08f4856a959dcf76ad509da14ae637f"
server_key_iv: "5398fbcea8719c547cb962aa3419b521"
server_key_version: "encryptionkey@0"
I20260808 03:51:43.868417 2742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:43.884871 2742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:43.885635 2742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:43.885890 2742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:43.903254 2742 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:40115
I20260808 03:51:43.903275 2836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:40115 every 8 connection(s)
I20260808 03:51:43.904541 2742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:51:43.906736 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 2742
I20260808 03:51:43.906946 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:51:43.907779 27210 external_mini_cluster.cc:1428] Setting key caa5627c8373f6dd40ff23f03e844955
I20260808 03:51:43.908938 2837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:43.923112 2840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:43.909778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44114 (local address 127.26.146.190:40115)
0808 03:51:43.910867 (+ 1089us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:43.910882 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:43.910940 (+ 58us) server_negotiation.cc:407] Connection header received
0808 03:51:43.913407 (+ 2467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:43.913434 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:43.914011 (+ 577us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:43.914443 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:43.914973 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:43.916999 (+ 2026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:43.918199 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:43.919211 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:43.919393 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:43.920461 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:43.920473 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:43.920495 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:51:43.921219 (+ 724us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:43.921325 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:51:43.921353 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:43.921466 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:43.921585 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 03:51:43.921893 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":357,"threads_started":1}
I20260808 03:51:43.923513 2837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a28d12d388847a7a93476f0562a203d: Bootstrap starting.
I20260808 03:51:43.926954 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:51:43.927181 2837 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a28d12d388847a7a93476f0562a203d: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:43.928398 2837 log.cc:824] T 00000000000000000000000000000000 P 1a28d12d388847a7a93476f0562a203d: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:43.931246 2837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a28d12d388847a7a93476f0562a203d: No bootstrap required, opened a new log
I20260808 03:51:43.938402 2837 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a28d12d388847a7a93476f0562a203d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a28d12d388847a7a93476f0562a203d" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 40115 } }
I20260808 03:51:43.938817 2837 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a28d12d388847a7a93476f0562a203d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:43.938918 2837 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a28d12d388847a7a93476f0562a203d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a28d12d388847a7a93476f0562a203d, State: Initialized, Role: FOLLOWER
I20260808 03:51:43.939419 2837 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a28d12d388847a7a93476f0562a203d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a28d12d388847a7a93476f0562a203d" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 40115 } }
I20260808 03:51:43.939637 2837 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a28d12d388847a7a93476f0562a203d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:51:43.939765 2837 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a28d12d388847a7a93476f0562a203d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:51:43.939982 2837 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a28d12d388847a7a93476f0562a203d [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:43.942071 2837 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a28d12d388847a7a93476f0562a203d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a28d12d388847a7a93476f0562a203d" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 40115 } }
I20260808 03:51:43.942554 2837 leader_election.cc:302] T 00000000000000000000000000000000 P 1a28d12d388847a7a93476f0562a203d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a28d12d388847a7a93476f0562a203d; no voters:
I20260808 03:51:43.943130 2837 leader_election.cc:288] T 00000000000000000000000000000000 P 1a28d12d388847a7a93476f0562a203d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:51:43.943261 2843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a28d12d388847a7a93476f0562a203d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:43.943979 2843 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a28d12d388847a7a93476f0562a203d [term 1 LEADER]: Becoming Leader. State: Replica: 1a28d12d388847a7a93476f0562a203d, State: Running, Role: LEADER
I20260808 03:51:43.944589 2843 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a28d12d388847a7a93476f0562a203d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a28d12d388847a7a93476f0562a203d" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 40115 } }
I20260808 03:51:43.945374 2837 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a28d12d388847a7a93476f0562a203d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:51:43.947388 2844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a28d12d388847a7a93476f0562a203d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a28d12d388847a7a93476f0562a203d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a28d12d388847a7a93476f0562a203d" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 40115 } } }
I20260808 03:51:43.947782 2844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a28d12d388847a7a93476f0562a203d [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:43.947978 2845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a28d12d388847a7a93476f0562a203d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a28d12d388847a7a93476f0562a203d. Latest consensus state: current_term: 1 leader_uuid: "1a28d12d388847a7a93476f0562a203d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a28d12d388847a7a93476f0562a203d" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 40115 } } }
I20260808 03:51:43.948308 2845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a28d12d388847a7a93476f0562a203d [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:43.955747 2853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:51:43.960652 2853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:51:43.969892 2853 catalog_manager.cc:1347] Generated new cluster ID: ef4e6ad11fda4eae8c7c9b1338c0fe71
I20260808 03:51:43.970003 2853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:51:43.980474 2853 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:51:43.981657 2853 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:51:43.987287 2853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a28d12d388847a7a93476f0562a203d: Generated new TSK 0
I20260808 03:51:43.988058 2853 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:51:44.014683 2840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:43.999732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44120 (local address 127.26.146.190:40115)
0808 03:51:43.999921 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:43.999927 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:44.000389 (+ 462us) server_negotiation.cc:407] Connection header received
0808 03:51:44.000627 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:44.000631 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:44.000699 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:44.000834 (+ 135us) server_negotiation.cc:226] Negotiated authn=JWT
0808 03:51:44.001286 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:44.002024 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:44.003353 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:44.003883 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:44.003993 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:44.004444 (+ 451us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 03:51:44.014029 (+ 9585us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 03:51:44.014369 (+ 340us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44120: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 03:51:44.014832 2840 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44120: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 03:51:44.014878 2873 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:51:43.999685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:40115 (local address 127.0.0.1:44120)
0808 03:51:44.000284 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:44.000299 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:44.000439 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:44.000866 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:44.000871 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:44.000883 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0808 03:51:44.001142 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:44.001152 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:44.002165 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:44.002171 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:44.003217 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:44.003229 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:44.004046 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:44.004054 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:44.004182 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:44.004215 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 03:51:44.014247 (+ 10032us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 03:51:44.014494 (+ 247us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.146.190:40115: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":340,"threads_started":1}
I20260808 03:51:44.015878 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 2742
I20260808 03:51:44.017057 2830 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:44.108414 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 2742
2026-08-08T03:51:44Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (692 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (407 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (417 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:51:45 dist-test-slave-jlzn krb5kdc[2891](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:51:45 dist-test-slave-jlzn krb5kdc[2891](info): set up 2 sockets
Aug 08 03:51:45 dist-test-slave-jlzn krb5kdc[2891](info): commencing operation
krb5kdc: starting...
W20260808 03:51:46.115710 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:51:46 dist-test-slave-jlzn krb5kdc[2891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:51:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:51:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:51:46.346691 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:42045
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:34349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:42045
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:46.667109 2910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:46.667379 2910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:46.667518 2910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:46.674597 2910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:51:46.674692 2910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:46.674767 2910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:46.674846 2910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:51:46.674906 2910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:51:46.684504 2910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:34349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:42045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:42045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:46.685741 2910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:46.687016 2910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:46.698241 2920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:46.698421 2922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:46.698747 2919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:46.698930 2910 server_base.cc:1034] running on GCE node
I20260808 03:51:46.699676 2910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:46.700959 2910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:46.702160 2910 hybrid_clock.cc:648] HybridClock initialized: now 1786161106702106 us; error 44 us; skew 500 ppm
I20260808 03:51:46.702486 2910 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:46 dist-test-slave-jlzn krb5kdc[2891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161106, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:46.706449 2910 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:51:46.707957 2910 webserver.cc:460] Webserver started at http://127.26.146.190:36499/ using document root <none> and password file <none>
I20260808 03:51:46.708608 2910 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:46.708734 2910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:46.709029 2910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:46.712131 2910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "666c853541134719a35be1c30d8a3584"
format_stamp: "Formatted at 2026-08-08 03:51:46 on dist-test-slave-jlzn"
server_key: "4172d671fd1ded995122ae1b8e1d858e"
server_key_iv: "cc974e2c54624340d19ff74f08760041"
server_key_version: "encryptionkey@0"
I20260808 03:51:46.712930 2910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "666c853541134719a35be1c30d8a3584"
format_stamp: "Formatted at 2026-08-08 03:51:46 on dist-test-slave-jlzn"
server_key: "4172d671fd1ded995122ae1b8e1d858e"
server_key_iv: "cc974e2c54624340d19ff74f08760041"
server_key_version: "encryptionkey@0"
I20260808 03:51:46.718235 2910 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260808 03:51:46.721714 2933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:46.723011 2910 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:51:46.723196 2910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "666c853541134719a35be1c30d8a3584"
format_stamp: "Formatted at 2026-08-08 03:51:46 on dist-test-slave-jlzn"
server_key: "4172d671fd1ded995122ae1b8e1d858e"
server_key_iv: "cc974e2c54624340d19ff74f08760041"
server_key_version: "encryptionkey@0"
I20260808 03:51:46.723382 2910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:46.755067 2910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:46.759461 2910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:46.759745 2910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:46.777311 2910 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:42045
I20260808 03:51:46.777343 3013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:42045 every 8 connection(s)
I20260808 03:51:46.778698 2910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:51:46.782298 3014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:46.787528 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 2910
I20260808 03:51:46.787714 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:51:46.788370 27210 external_mini_cluster.cc:1428] Setting key 6b58fc5bd737c7b37b088431a437afa4
I20260808 03:51:46.793656 3014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 666c853541134719a35be1c30d8a3584: Bootstrap starting.
I20260808 03:51:46.797317 3014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 666c853541134719a35be1c30d8a3584: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:46.798456 3014 log.cc:824] T 00000000000000000000000000000000 P 666c853541134719a35be1c30d8a3584: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:46.801514 3014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 666c853541134719a35be1c30d8a3584: No bootstrap required, opened a new log
Aug 08 03:51:46 dist-test-slave-jlzn krb5kdc[2891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:46.810343 3014 raft_consensus.cc:348] T 00000000000000000000000000000000 P 666c853541134719a35be1c30d8a3584 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "666c853541134719a35be1c30d8a3584" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 42045 } }
I20260808 03:51:46.810815 3014 raft_consensus.cc:374] T 00000000000000000000000000000000 P 666c853541134719a35be1c30d8a3584 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:46.810920 3014 raft_consensus.cc:729] T 00000000000000000000000000000000 P 666c853541134719a35be1c30d8a3584 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666c853541134719a35be1c30d8a3584, State: Initialized, Role: FOLLOWER
I20260808 03:51:46.811472 3014 consensus_queue.cc:260] T 00000000000000000000000000000000 P 666c853541134719a35be1c30d8a3584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "666c853541134719a35be1c30d8a3584" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 42045 } }
I20260808 03:51:46.811733 3014 raft_consensus.cc:388] T 00000000000000000000000000000000 P 666c853541134719a35be1c30d8a3584 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:51:46.811900 3017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:46.790368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48482 (local address 127.26.146.190:42045)
0808 03:51:46.791215 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:46.791231 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:46.791293 (+ 62us) server_negotiation.cc:407] Connection header received
0808 03:51:46.793644 (+ 2351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:46.793673 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:46.794162 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:46.794578 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:46.795935 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:46.797887 (+ 1952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:46.799238 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:46.800088 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:46.800270 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:46.804552 (+ 4282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:46.804560 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:46.804574 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:46.804618 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:46.807643 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:46.808499 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:46.808505 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:46.808514 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:46.808695 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:46.809068 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:46.809071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:46.809073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:46.809617 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:46.809824 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:46.810382 (+ 558us) server_negotiation.cc:299] Negotiation successful
0808 03:51:46.810741 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":358,"threads_started":1}
I20260808 03:51:46.811933 3014 raft_consensus.cc:482] T 00000000000000000000000000000000 P 666c853541134719a35be1c30d8a3584 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:51:46.812137 3014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 666c853541134719a35be1c30d8a3584 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:46.814386 3014 raft_consensus.cc:504] T 00000000000000000000000000000000 P 666c853541134719a35be1c30d8a3584 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "666c853541134719a35be1c30d8a3584" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 42045 } }
I20260808 03:51:46.814852 3014 leader_election.cc:302] T 00000000000000000000000000000000 P 666c853541134719a35be1c30d8a3584 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 666c853541134719a35be1c30d8a3584; no voters:
I20260808 03:51:46.815392 3014 leader_election.cc:288] T 00000000000000000000000000000000 P 666c853541134719a35be1c30d8a3584 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:51:46.815501 3021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 666c853541134719a35be1c30d8a3584 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:46.816576 3021 raft_consensus.cc:686] T 00000000000000000000000000000000 P 666c853541134719a35be1c30d8a3584 [term 1 LEADER]: Becoming Leader. State: Replica: 666c853541134719a35be1c30d8a3584, State: Running, Role: LEADER
I20260808 03:51:46.817126 3021 consensus_queue.cc:237] T 00000000000000000000000000000000 P 666c853541134719a35be1c30d8a3584 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "666c853541134719a35be1c30d8a3584" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 42045 } }
I20260808 03:51:46.817386 3014 sys_catalog.cc:564] T 00000000000000000000000000000000 P 666c853541134719a35be1c30d8a3584 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:51:46.819854 3025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666c853541134719a35be1c30d8a3584 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 666c853541134719a35be1c30d8a3584. Latest consensus state: current_term: 1 leader_uuid: "666c853541134719a35be1c30d8a3584" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "666c853541134719a35be1c30d8a3584" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 42045 } } }
I20260808 03:51:46.820312 3025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666c853541134719a35be1c30d8a3584 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:46.819880 3023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666c853541134719a35be1c30d8a3584 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "666c853541134719a35be1c30d8a3584" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "666c853541134719a35be1c30d8a3584" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 42045 } } }
I20260808 03:51:46.820899 3023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666c853541134719a35be1c30d8a3584 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:46.824826 3031 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:51:46.829193 3031 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:51:46.836719 3031 catalog_manager.cc:1347] Generated new cluster ID: 124801dc5d304c2785f9cdd52cb00120
I20260808 03:51:46.836828 3031 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:51:46.862923 3031 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:51:46.864888 3031 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:51:46.879940 3031 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 666c853541134719a35be1c30d8a3584: Generated new TSK 0
I20260808 03:51:46.880791 3031 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
I20260808 03:51:46.948220 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:42045
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:34349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:47.276252 3054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:47.276526 3054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:47.276659 3054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:47.283659 3054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:47.283769 3054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:47.283927 3054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:51:47.293324 3054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:34349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:42045
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:47.294538 3054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:47.295843 3054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:47.308338 3062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:47.308265 3061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:47.308372 3065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:47.309088 3054 server_base.cc:1034] running on GCE node
I20260808 03:51:47.309643 3054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:47.310523 3054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:47.311723 3054 hybrid_clock.cc:648] HybridClock initialized: now 1786161107311670 us; error 40 us; skew 500 ppm
I20260808 03:51:47.312037 3054 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:47 dist-test-slave-jlzn krb5kdc[2891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161107, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:47.315126 3054 init.cc:377] Logged in from keytab as kudu/127.26.146.129@KRBTEST.COM (short username kudu)
I20260808 03:51:47.316486 3054 webserver.cc:460] Webserver started at http://127.26.146.129:42467/ using document root <none> and password file <none>
I20260808 03:51:47.317129 3054 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:47.317248 3054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:47.317553 3054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:47.320436 3054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "07b41cabe42845e09ef5d9ca3c6633b0"
format_stamp: "Formatted at 2026-08-08 03:51:47 on dist-test-slave-jlzn"
server_key: "e577e863e76fe0adef7fc1a631a56f40"
server_key_iv: "b5f9297df29aa7048513ea3b164d6702"
server_key_version: "encryptionkey@0"
I20260808 03:51:47.321178 3054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "07b41cabe42845e09ef5d9ca3c6633b0"
format_stamp: "Formatted at 2026-08-08 03:51:47 on dist-test-slave-jlzn"
server_key: "e577e863e76fe0adef7fc1a631a56f40"
server_key_iv: "b5f9297df29aa7048513ea3b164d6702"
server_key_version: "encryptionkey@0"
I20260808 03:51:47.326265 3054 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:51:47.329665 3075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:47.330920 3054 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260808 03:51:47.331070 3054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "07b41cabe42845e09ef5d9ca3c6633b0"
format_stamp: "Formatted at 2026-08-08 03:51:47 on dist-test-slave-jlzn"
server_key: "e577e863e76fe0adef7fc1a631a56f40"
server_key_iv: "b5f9297df29aa7048513ea3b164d6702"
server_key_version: "encryptionkey@0"
I20260808 03:51:47.331246 3054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:47.346640 3054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:47.350256 3054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:47.350510 3054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:47.351390 3054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:47.352684 3054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:47.352769 3054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:47.352893 3054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:47.352948 3054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:47.383270 3054 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:42513
I20260808 03:51:47.383311 3241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:42513 every 8 connection(s)
I20260808 03:51:47.384526 3054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:51:47.391930 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 3054
I20260808 03:51:47.392117 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:51:47.392825 27210 external_mini_cluster.cc:1428] Setting key cf5dc249cd45ca87c555eb8c1b8f456a
Aug 08 03:51:47 dist-test-slave-jlzn krb5kdc[2891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161107, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:47.408732 3244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:47.386868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:55413 (local address 127.26.146.190:42045)
0808 03:51:47.387285 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:47.387289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:47.388173 (+ 884us) server_negotiation.cc:407] Connection header received
0808 03:51:47.389940 (+ 1767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:47.389946 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:47.390041 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:47.390145 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:47.391624 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:47.392466 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:47.394287 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:47.394885 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:47.394983 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:47.402810 (+ 7827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:47.402815 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:47.402818 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:47.402849 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:47.405558 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:47.406246 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:47.406251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:47.406254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:47.406394 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:47.406876 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:47.406881 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:47.406883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:47.407228 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:47.407376 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:47.408137 (+ 761us) server_negotiation.cc:299] Negotiation successful
0808 03:51:47.408380 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":214,"threads_started":1}
I20260808 03:51:47.409544 3245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:47.387292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:42045 (local address 127.26.146.129:55413)
0808 03:51:47.388001 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:47.388044 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:47.389680 (+ 1636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:47.390379 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:47.390386 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:47.390883 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:47.391418 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:47.391436 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:47.392641 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:47.392647 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:47.394152 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:47.394165 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:47.395112 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:47.395120 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:47.395406 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:47.400007 (+ 4601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:47.400038 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:47.402577 (+ 2539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:47.405706 (+ 3129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:47.405713 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:47.405725 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:47.406125 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:47.406541 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:47.406545 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:47.406548 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:47.406753 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:47.407340 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:47.407345 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:47.407725 (+ 380us) client_negotiation.cc:769] Sending connection context
0808 03:51:47.408024 (+ 299us) client_negotiation.cc:240] Negotiation successful
0808 03:51:47.408416 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":265,"threads_started":1}
I20260808 03:51:47.411053 3242 heartbeater.cc:347] Connected to a master server at 127.26.146.190:42045
I20260808 03:51:47.411453 3242 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:47.412339 3242 heartbeater.cc:511] Master 127.26.146.190:42045 requested a full tablet report, sending...
I20260808 03:51:47.414649 2959 ts_manager.cc:194] Registered new tserver with Master: 07b41cabe42845e09ef5d9ca3c6633b0 (127.26.146.129:42513)
I20260808 03:51:47.416162 2959 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.129@KRBTEST.COM'} at 127.26.146.129:55413
WARNING: no policy specified for kudu/127.26.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
I20260808 03:51:47.446759 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:42045
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:34349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:47.769671 3251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:47.769956 3251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:47.770088 3251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:47.777652 3251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:47.777767 3251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:47.777938 3251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:51:47.787532 3251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:34349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:42045
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:47.788707 3251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:47.789948 3251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:47.802501 3261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:47.802582 3257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:47.802556 3258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:47.803644 3251 server_base.cc:1034] running on GCE node
I20260808 03:51:47.804255 3251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:47.805130 3251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:47.806334 3251 hybrid_clock.cc:648] HybridClock initialized: now 1786161107806279 us; error 47 us; skew 500 ppm
I20260808 03:51:47.806625 3251 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:47 dist-test-slave-jlzn krb5kdc[2891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161107, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:47.809854 3251 init.cc:377] Logged in from keytab as kudu/127.26.146.130@KRBTEST.COM (short username kudu)
I20260808 03:51:47.811153 3251 webserver.cc:460] Webserver started at http://127.26.146.130:42235/ using document root <none> and password file <none>
I20260808 03:51:47.811861 3251 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:47.811988 3251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:47.812283 3251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:47.815128 3251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "ce857dcd60144470af01f9b4d2d61770"
format_stamp: "Formatted at 2026-08-08 03:51:47 on dist-test-slave-jlzn"
server_key: "71a4a4e89919e219da90831e27bea621"
server_key_iv: "63ee2cb60427afb36842f9f6ae19d909"
server_key_version: "encryptionkey@0"
I20260808 03:51:47.815984 3251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "ce857dcd60144470af01f9b4d2d61770"
format_stamp: "Formatted at 2026-08-08 03:51:47 on dist-test-slave-jlzn"
server_key: "71a4a4e89919e219da90831e27bea621"
server_key_iv: "63ee2cb60427afb36842f9f6ae19d909"
server_key_version: "encryptionkey@0"
I20260808 03:51:47.821048 3251 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 03:51:47.824473 3272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:47.825730 3251 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:51:47.825886 3251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "ce857dcd60144470af01f9b4d2d61770"
format_stamp: "Formatted at 2026-08-08 03:51:47 on dist-test-slave-jlzn"
server_key: "71a4a4e89919e219da90831e27bea621"
server_key_iv: "63ee2cb60427afb36842f9f6ae19d909"
server_key_version: "encryptionkey@0"
I20260808 03:51:47.826077 3251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:47.840513 3251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:47.844544 3251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:47.844810 3251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:47.845731 3251 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:47.846980 3251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:47.847059 3251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:47.847187 3251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:47.847249 3251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:47.876824 3251 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:38077
I20260808 03:51:47.876884 3430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:38077 every 8 connection(s)
I20260808 03:51:47.878136 3251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:51:47.887418 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 3251
I20260808 03:51:47.887607 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:51:47.888214 27210 external_mini_cluster.cc:1428] Setting key 5b8e8ec2b333c833f0baa9340d948c0b
Aug 08 03:51:47 dist-test-slave-jlzn krb5kdc[2891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161107, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:47.898401 3244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:47.880619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:59479 (local address 127.26.146.190:42045)
0808 03:51:47.880817 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:47.880823 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:47.882006 (+ 1183us) server_negotiation.cc:407] Connection header received
0808 03:51:47.883879 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:47.883885 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:47.883962 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:47.884047 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:47.885496 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:47.886273 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:47.888168 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:47.888728 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:47.888843 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:47.892515 (+ 3672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:47.892522 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:47.892526 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:47.892565 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:47.895443 (+ 2878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:47.896164 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:47.896169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:47.896172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:47.896339 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:47.896730 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:47.896734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:47.896737 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:47.897081 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:47.897226 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:47.897861 (+ 635us) server_negotiation.cc:299] Negotiation successful
0808 03:51:47.898075 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 03:51:47.899251 3434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:47.881147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:42045 (local address 127.26.146.130:59479)
0808 03:51:47.881848 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:47.881894 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:47.883546 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:47.884281 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:47.884290 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:47.884784 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:47.885356 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:47.885374 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:47.886387 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:47.886391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:47.888014 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:47.888030 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:47.888986 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:47.888995 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:47.889288 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:47.889988 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:47.890010 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:47.892281 (+ 2271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:47.895572 (+ 3291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:47.895579 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:47.895591 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:47.896055 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:47.896436 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:47.896439 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:47.896441 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:47.896624 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:47.897181 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:47.897186 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:47.897513 (+ 327us) client_negotiation.cc:769] Sending connection context
0808 03:51:47.897729 (+ 216us) client_negotiation.cc:240] Negotiation successful
0808 03:51:47.898110 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":270,"threads_started":1}
I20260808 03:51:47.900681 3431 heartbeater.cc:347] Connected to a master server at 127.26.146.190:42045
I20260808 03:51:47.900978 3431 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:47.901679 3431 heartbeater.cc:511] Master 127.26.146.190:42045 requested a full tablet report, sending...
I20260808 03:51:47.903179 2959 ts_manager.cc:194] Registered new tserver with Master: ce857dcd60144470af01f9b4d2d61770 (127.26.146.130:38077)
I20260808 03:51:47.904137 2959 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.130@KRBTEST.COM'} at 127.26.146.130:59479
WARNING: no policy specified for kudu/127.26.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
I20260808 03:51:47.943398 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:42045
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:34349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:48.268610 3437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:48.268918 3437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:48.269042 3437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:48.276291 3437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:48.276399 3437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:48.276544 3437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:51:48.286307 3437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:34349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:42045
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:48.287585 3437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:48.288908 3437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:48.301539 3444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:48.301556 3445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:48.301771 3447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:48.302134 3437 server_base.cc:1034] running on GCE node
I20260808 03:51:48.302760 3437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:48.303625 3437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:48.304822 3437 hybrid_clock.cc:648] HybridClock initialized: now 1786161108304768 us; error 56 us; skew 500 ppm
I20260808 03:51:48.305109 3437 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:48 dist-test-slave-jlzn krb5kdc[2891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161108, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:48.308084 3437 init.cc:377] Logged in from keytab as kudu/127.26.146.131@KRBTEST.COM (short username kudu)
I20260808 03:51:48.309430 3437 webserver.cc:460] Webserver started at http://127.26.146.131:32777/ using document root <none> and password file <none>
I20260808 03:51:48.310083 3437 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:48.310209 3437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:48.310501 3437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:48.313429 3437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "be8ee37411c64fcda5260459659e623b"
format_stamp: "Formatted at 2026-08-08 03:51:48 on dist-test-slave-jlzn"
server_key: "21ffb753d4bf321fb5620453d92dc23a"
server_key_iv: "7598f0ee22ba91bea03158910834a62f"
server_key_version: "encryptionkey@0"
I20260808 03:51:48.314172 3437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "be8ee37411c64fcda5260459659e623b"
format_stamp: "Formatted at 2026-08-08 03:51:48 on dist-test-slave-jlzn"
server_key: "21ffb753d4bf321fb5620453d92dc23a"
server_key_iv: "7598f0ee22ba91bea03158910834a62f"
server_key_version: "encryptionkey@0"
I20260808 03:51:48.319227 3437 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260808 03:51:48.322480 3457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:48.323877 3437 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 03:51:48.324031 3437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "be8ee37411c64fcda5260459659e623b"
format_stamp: "Formatted at 2026-08-08 03:51:48 on dist-test-slave-jlzn"
server_key: "21ffb753d4bf321fb5620453d92dc23a"
server_key_iv: "7598f0ee22ba91bea03158910834a62f"
server_key_version: "encryptionkey@0"
I20260808 03:51:48.324223 3437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:48.338657 3437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:48.342393 3437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:48.342664 3437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:48.343632 3437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:48.345005 3437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:48.345088 3437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:48.345206 3437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:48.345263 3437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:48.376065 3437 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:35039
I20260808 03:51:48.376101 3617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:35039 every 8 connection(s)
I20260808 03:51:48.377351 3437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:51:48.384099 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 3437
I20260808 03:51:48.384279 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:51:48.384846 27210 external_mini_cluster.cc:1428] Setting key 0bd59d79fe9518359f482e79f307e810
Aug 08 03:51:48 dist-test-slave-jlzn krb5kdc[2891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161108, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:51:48.397459 3244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:48.379930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:48551 (local address 127.26.146.190:42045)
0808 03:51:48.380101 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:48.380106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:48.381284 (+ 1178us) server_negotiation.cc:407] Connection header received
0808 03:51:48.383152 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:48.383158 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:48.383255 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:48.383345 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:48.385344 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:48.386348 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:48.388530 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:48.388935 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:48.389026 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:48.392280 (+ 3254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:48.392284 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:48.392287 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:48.392318 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:48.394586 (+ 2268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:48.395258 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:48.395263 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:48.395266 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:48.395395 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:48.395825 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:48.395831 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:48.395834 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:48.396104 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:48.396243 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:48.396899 (+ 656us) server_negotiation.cc:299] Negotiation successful
0808 03:51:48.397117 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 03:51:48.398413 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:48.380375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:42045 (local address 127.26.146.131:48551)
0808 03:51:48.381101 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:48.381142 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:48.382815 (+ 1673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:48.383594 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:48.383602 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:48.384354 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:48.385113 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:48.385141 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:48.386503 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:48.386509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:48.388385 (+ 1876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:48.388398 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:48.389101 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:48.389106 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:48.389298 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:48.389895 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:48.389917 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:48.392066 (+ 2149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:48.394727 (+ 2661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:48.394736 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:48.394749 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:48.395131 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:48.395496 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:48.395501 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:48.395504 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:48.395687 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:48.396224 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:48.396231 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:48.396535 (+ 304us) client_negotiation.cc:769] Sending connection context
0808 03:51:48.396762 (+ 227us) client_negotiation.cc:240] Negotiation successful
0808 03:51:48.397244 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":302,"threads_started":1}
I20260808 03:51:48.399792 3618 heartbeater.cc:347] Connected to a master server at 127.26.146.190:42045
I20260808 03:51:48.400144 3618 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:48.400866 3618 heartbeater.cc:511] Master 127.26.146.190:42045 requested a full tablet report, sending...
I20260808 03:51:48.402185 2959 ts_manager.cc:194] Registered new tserver with Master: be8ee37411c64fcda5260459659e623b (127.26.146.131:35039)
I20260808 03:51:48.402953 2959 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.131@KRBTEST.COM'} at 127.26.146.131:48551
I20260808 03:51:48.412192 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:51:48.419281 3242 heartbeater.cc:503] Master 127.26.146.190:42045 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/test-admin.keytab.
Aug 08 03:51:48 dist-test-slave-jlzn krb5kdc[2891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:48.443003 27210 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 03:51:48.448144 3628 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:51:48.445642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44650 (local address 127.26.146.129:42513)
0808 03:51:48.446273 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:48.446283 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:48.446316 (+ 33us) server_negotiation.cc:407] Connection header received
0808 03:51:48.446381 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:48.446388 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:48.446596 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:48.446752 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:48.447682 (+ 930us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44650: BlockingRecv error: recv got EOF from 127.0.0.1:44650 (error 108)
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":281,"threads_started":1}
W20260808 03:51:48.448166 27210 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/krb5cc' of type FILE
Aug 08 03:51:48 dist-test-slave-jlzn krb5kdc[2891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:48.449647 27210 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 03:51:48 dist-test-slave-jlzn krb5kdc[2891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:48.450889 27210 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 03:51:48 dist-test-slave-jlzn krb5kdc[2891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.129@KRBTEST.COM
I20260808 03:51:48.465818 3628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:48.452404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44654 (local address 127.26.146.129:42513)
0808 03:51:48.452574 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:48.452580 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:48.452856 (+ 276us) server_negotiation.cc:407] Connection header received
0808 03:51:48.453017 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:48.453021 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:48.453078 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:48.453157 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:48.454174 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:48.455147 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:48.456416 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:48.456989 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:48.457106 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:48.460532 (+ 3426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:48.460544 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:48.460558 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:48.460598 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:48.463083 (+ 2485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:48.463720 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:48.463734 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:48.463736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:48.463926 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:48.464301 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:48.464306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:48.464310 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:48.464680 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:48.464971 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:48.465292 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 03:51:48.465554 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 03:51:48.471048 3628 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:51:48.469551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44658 (local address 127.26.146.129:42513)
0808 03:51:48.469746 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:48.469752 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:48.469880 (+ 128us) server_negotiation.cc:407] Connection header received
0808 03:51:48.470022 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:48.470027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:48.470100 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:48.470176 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:48.470849 (+ 673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44658: BlockingRecv error: recv got EOF from 127.0.0.1:44658 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 03:51:48.471203 27210 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/krb5cc' of type FILE
Aug 08 03:51:48 dist-test-slave-jlzn krb5kdc[2891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:48.472607 27210 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 03:51:48 dist-test-slave-jlzn krb5kdc[2891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:48.473757 27210 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 03:51:48 dist-test-slave-jlzn krb5kdc[2891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.129@KRBTEST.COM
I20260808 03:51:48.486354 3628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:48.474887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44668 (local address 127.26.146.129:42513)
0808 03:51:48.475046 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:48.475051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:48.475311 (+ 260us) server_negotiation.cc:407] Connection header received
0808 03:51:48.475479 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:48.475484 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:48.475536 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:48.475610 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:48.476543 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:48.477220 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:48.478393 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:48.478791 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:48.478905 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:48.482190 (+ 3285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:48.482194 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:48.482198 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:48.482228 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:48.484420 (+ 2192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:48.484956 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:48.484959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:48.484962 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:48.485066 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:48.485348 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:48.485351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:48.485353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:48.485580 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:48.485778 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:48.485940 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 03:51:48.486084 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 03:51:48.490213 3628 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:51:48.488805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44684 (local address 127.26.146.129:42513)
0808 03:51:48.488977 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:48.488982 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:48.489074 (+ 92us) server_negotiation.cc:407] Connection header received
0808 03:51:48.489180 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:48.489183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:48.489248 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:48.489320 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:48.490023 (+ 703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44684: BlockingRecv error: recv got EOF from 127.0.0.1:44684 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260808 03:51:48.490387 27210 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786161058098901-27210-0/krb5kdc/krb5cc' of type FILE
Aug 08 03:51:48 dist-test-slave-jlzn krb5kdc[2891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:48.491755 27210 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 03:51:48 dist-test-slave-jlzn krb5kdc[2891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:48.492961 27210 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 03:51:48 dist-test-slave-jlzn krb5kdc[2891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.129@KRBTEST.COM
I20260808 03:51:48.505985 3628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:48.494188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44692 (local address 127.26.146.129:42513)
0808 03:51:48.494357 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:48.494362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:48.494528 (+ 166us) server_negotiation.cc:407] Connection header received
0808 03:51:48.494668 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:48.494671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:48.494721 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:48.494791 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:48.495691 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:48.496564 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:48.497673 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:48.498087 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:48.498175 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:48.501299 (+ 3124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:48.501303 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:48.501306 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:48.501337 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:48.503704 (+ 2367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:48.504273 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:48.504277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:48.504279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:48.504428 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:48.504785 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:48.504788 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:48.504790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:48.505098 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:48.505376 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:48.505557 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 03:51:48.505719 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 03:51:48.507457 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 3054
I20260808 03:51:48.518083 3628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:48.508109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44706 (local address 127.26.146.129:42513)
0808 03:51:48.508242 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:48.508245 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:48.508551 (+ 306us) server_negotiation.cc:407] Connection header received
0808 03:51:48.508673 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:48.508677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:48.508735 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:48.508807 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:48.509663 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:48.510311 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:48.511392 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:48.511828 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:48.511920 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:48.513650 (+ 1730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:48.513654 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:48.513657 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:48.513684 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:48.516007 (+ 2323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:48.516557 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:48.516561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:48.516563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:48.516693 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:48.516952 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:48.516955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:48.516957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:48.517182 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:48.517383 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:48.517590 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 03:51:48.517814 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 03:51:48.518710 3236 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:48.600443 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 3054
I20260808 03:51:48.622493 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 3251
Aug 08 03:51:48 dist-test-slave-jlzn krb5kdc[2891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.130@KRBTEST.COM
I20260808 03:51:48.637732 3647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:48.623398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33966 (local address 127.26.146.130:38077)
0808 03:51:48.623976 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:48.623983 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:48.624017 (+ 34us) server_negotiation.cc:407] Connection header received
0808 03:51:48.624082 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:48.624089 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:48.624297 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:48.624454 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:48.625471 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:48.626562 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:48.627889 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:48.628431 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:48.628548 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:48.632611 (+ 4063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:48.632622 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:48.632636 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:48.632670 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:48.635310 (+ 2640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:48.636035 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:48.636042 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:48.636045 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:48.636170 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:48.636485 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:48.636488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:48.636490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:48.636810 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:48.637031 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:48.637255 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 03:51:48.637443 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":258,"threads_started":1}
I20260808 03:51:48.638988 3424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:48.725832 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 3251
I20260808 03:51:48.749646 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 3437
Aug 08 03:51:48 dist-test-slave-jlzn krb5kdc[2891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.131@KRBTEST.COM
I20260808 03:51:48.768788 3649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:48.750638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33782 (local address 127.26.146.131:35039)
0808 03:51:48.751327 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:48.751338 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:48.751373 (+ 35us) server_negotiation.cc:407] Connection header received
0808 03:51:48.751454 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:48.751462 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:48.751798 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:48.752099 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:48.753501 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:48.755121 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:48.756676 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:48.757258 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:48.757374 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:48.762594 (+ 5220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:48.762607 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:48.762624 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:48.762661 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:48.765803 (+ 3142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:48.766654 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:48.766661 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:48.766663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:48.766788 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:48.767207 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:48.767210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:48.767212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:48.767563 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:48.767790 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:48.768203 (+ 413us) server_negotiation.cc:299] Negotiation successful
0808 03:51:48.768436 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":306,"threads_started":1}
I20260808 03:51:48.770087 3613 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:48.851444 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 3437
I20260808 03:51:48.872218 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 2910
I20260808 03:51:48.873487 3007 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:48.975795 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 2910
2026-08-08T03:51:48Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4048 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:51:49 dist-test-slave-jlzn krb5kdc[3656](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:51:49 dist-test-slave-jlzn krb5kdc[3656](info): set up 2 sockets
Aug 08 03:51:49 dist-test-slave-jlzn krb5kdc[3656](info): commencing operation
krb5kdc: starting...
W20260808 03:51:50.158776 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:51:50 dist-test-slave-jlzn krb5kdc[3656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:51:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:51:50Z Disabled control of system clock
WARNING: no policy specified for oryx/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.190.keytab.
Entry for principal oryx/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.190.keytab.
I20260808 03:51:50.393366 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:37325
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.190.keytab
--principal=oryx/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:40669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:37325
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:50.720212 3676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:50.720497 3676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:50.720652 3676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:50.728091 3676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:51:50.728201 3676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:50.728276 3676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:50.728346 3676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:51:50.728401 3676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:51:50.738401 3676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:40669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:37325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.190.keytab
--principal=oryx/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:37325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:50.739754 3676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:50.741050 3676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:50.753168 3684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:50.753242 3683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:50.753218 3687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:50.753794 3676 server_base.cc:1034] running on GCE node
I20260808 03:51:50.754566 3676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:50.755934 3676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:50.757136 3676 hybrid_clock.cc:648] HybridClock initialized: now 1786161110757078 us; error 49 us; skew 500 ppm
I20260808 03:51:50.757459 3676 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:50 dist-test-slave-jlzn krb5kdc[3656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161110, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:50.760613 3676 init.cc:377] Logged in from keytab as oryx/127.26.146.190@KRBTEST.COM (short username oryx)
I20260808 03:51:50.762005 3676 webserver.cc:460] Webserver started at http://127.26.146.190:46091/ using document root <none> and password file <none>
I20260808 03:51:50.762635 3676 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:50.762745 3676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:50.763002 3676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:50.765915 3676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "4ae9871e1feb40edb677f7bb5a2cb131"
format_stamp: "Formatted at 2026-08-08 03:51:50 on dist-test-slave-jlzn"
server_key: "04fa2cd7a8c0330d0d8a6191ef397aa3"
server_key_iv: "a3e71dad775b29ca61ff0d0e3ccbeb1b"
server_key_version: "encryptionkey@0"
I20260808 03:51:50.766616 3676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "4ae9871e1feb40edb677f7bb5a2cb131"
format_stamp: "Formatted at 2026-08-08 03:51:50 on dist-test-slave-jlzn"
server_key: "04fa2cd7a8c0330d0d8a6191ef397aa3"
server_key_iv: "a3e71dad775b29ca61ff0d0e3ccbeb1b"
server_key_version: "encryptionkey@0"
I20260808 03:51:50.771723 3676 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 03:51:50.775005 3695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:50.776335 3676 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 03:51:50.776494 3676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "4ae9871e1feb40edb677f7bb5a2cb131"
format_stamp: "Formatted at 2026-08-08 03:51:50 on dist-test-slave-jlzn"
server_key: "04fa2cd7a8c0330d0d8a6191ef397aa3"
server_key_iv: "a3e71dad775b29ca61ff0d0e3ccbeb1b"
server_key_version: "encryptionkey@0"
I20260808 03:51:50.776675 3676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:50.797814 3676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:50.801954 3676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:50.802213 3676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:50.819720 3676 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:37325
I20260808 03:51:50.819766 3768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:37325 every 8 connection(s)
I20260808 03:51:50.821151 3676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:51:50.823849 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 3676
I20260808 03:51:50.824048 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:51:50.824693 27210 external_mini_cluster.cc:1428] Setting key 2ed006fd82ea192727a04bbbc5135089
I20260808 03:51:50.825070 3770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:50.839154 3770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131: Bootstrap starting.
Aug 08 03:51:50 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM
I20260808 03:51:50.842671 3770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:50.843724 3770 log.cc:824] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:50.846895 3770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131: No bootstrap required, opened a new log
I20260808 03:51:50.847720 3774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:50.826623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43180 (local address 127.26.146.190:37325)
0808 03:51:50.827491 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:50.827506 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:50.827554 (+ 48us) server_negotiation.cc:407] Connection header received
0808 03:51:50.829906 (+ 2352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:50.829936 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:50.830461 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:50.830902 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:50.832316 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:50.834237 (+ 1921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:50.835437 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:50.836561 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:50.836753 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:50.840698 (+ 3945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:50.840716 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:50.840730 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:50.840772 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:50.843684 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:50.844551 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:50.844559 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:50.844567 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:50.844794 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:50.845144 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:50.845148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:50.845150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:50.845651 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:50.845891 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:50.846324 (+ 433us) server_negotiation.cc:299] Negotiation successful
0808 03:51:50.846655 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":386,"threads_started":1}
I20260808 03:51:50.854782 3770 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } }
I20260808 03:51:50.855118 3770 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:50.855222 3770 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ae9871e1feb40edb677f7bb5a2cb131, State: Initialized, Role: FOLLOWER
I20260808 03:51:50.855784 3770 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } }
I20260808 03:51:50.856033 3770 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:51:50.856174 3770 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:51:50.856333 3770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:50.858646 3770 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } }
I20260808 03:51:50.859135 3770 leader_election.cc:302] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ae9871e1feb40edb677f7bb5a2cb131; no voters:
I20260808 03:51:50.859648 3770 leader_election.cc:288] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:51:50.859769 3776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:50.860460 3776 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [term 1 LEADER]: Becoming Leader. State: Replica: 4ae9871e1feb40edb677f7bb5a2cb131, State: Running, Role: LEADER
I20260808 03:51:50.861019 3776 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } }
I20260808 03:51:50.861593 3770 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:51:50.864395 3777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } }
I20260808 03:51:50.864745 3777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:50.865280 3778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ae9871e1feb40edb677f7bb5a2cb131. Latest consensus state: current_term: 1 leader_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } }
I20260808 03:51:50.865597 3778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [sys.catalog]: This master's current role is: LEADER
W20260808 03:51:50.879021 3802 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 03:51:50.879148 3802 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 03:51:50.879269 3783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:51:50.881408 3783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:51:50.889360 3783 catalog_manager.cc:1347] Generated new cluster ID: 3040dace32494f9d811b8cb7d4d13dab
I20260808 03:51:50.889509 3783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:51:50.932231 3783 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:51:50.934177 3783 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:51:50.947440 3783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131: Generated new TSK 0
I20260808 03:51:50.948344 3783 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.26.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.129.keytab.
Entry for principal oryx/127.26.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.129.keytab.
I20260808 03:51:51.015595 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:37325
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.129.keytab
--principal=oryx/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:40669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:51.344653 3808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:51.344965 3808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:51.345099 3808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:51.352576 3808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:51.352706 3808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:51.352859 3808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:51:51.362769 3808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:40669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.129.keytab
--principal=oryx/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:37325
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:51.364166 3808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:51.365442 3808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:51.378365 3820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:51.378315 3819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:51.378774 3823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:51.379076 3808 server_base.cc:1034] running on GCE node
I20260808 03:51:51.379678 3808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:51.380645 3808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:51.381860 3808 hybrid_clock.cc:648] HybridClock initialized: now 1786161111381801 us; error 53 us; skew 500 ppm
I20260808 03:51:51.382143 3808 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:51 dist-test-slave-jlzn krb5kdc[3656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161111, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:51.385265 3808 init.cc:377] Logged in from keytab as oryx/127.26.146.129@KRBTEST.COM (short username oryx)
I20260808 03:51:51.386605 3808 webserver.cc:460] Webserver started at http://127.26.146.129:35721/ using document root <none> and password file <none>
I20260808 03:51:51.387243 3808 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:51.387343 3808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:51.387589 3808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:51.390846 3808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "502e92ad9ce04a7ba76a5e857a7e51d6"
format_stamp: "Formatted at 2026-08-08 03:51:51 on dist-test-slave-jlzn"
server_key: "1bd36488c5ef2a9d7dff75ad76925ebd"
server_key_iv: "ec7296d32b3a8cb64701b50bd2e05a1c"
server_key_version: "encryptionkey@0"
I20260808 03:51:51.391577 3808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "502e92ad9ce04a7ba76a5e857a7e51d6"
format_stamp: "Formatted at 2026-08-08 03:51:51 on dist-test-slave-jlzn"
server_key: "1bd36488c5ef2a9d7dff75ad76925ebd"
server_key_iv: "ec7296d32b3a8cb64701b50bd2e05a1c"
server_key_version: "encryptionkey@0"
I20260808 03:51:51.396428 3808 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 03:51:51.399729 3833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:51.400982 3808 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 03:51:51.401137 3808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "502e92ad9ce04a7ba76a5e857a7e51d6"
format_stamp: "Formatted at 2026-08-08 03:51:51 on dist-test-slave-jlzn"
server_key: "1bd36488c5ef2a9d7dff75ad76925ebd"
server_key_iv: "ec7296d32b3a8cb64701b50bd2e05a1c"
server_key_version: "encryptionkey@0"
I20260808 03:51:51.401315 3808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:51.417603 3808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:51.421628 3808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:51.421900 3808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:51.422789 3808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:51.424074 3808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:51.424152 3808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:51.424293 3808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:51.424346 3808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:51.453826 3808 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:38971
I20260808 03:51:51.453850 3993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:38971 every 8 connection(s)
I20260808 03:51:51.455039 3808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:51:51.459957 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 3808
I20260808 03:51:51.460237 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:51:51.461066 27210 external_mini_cluster.cc:1428] Setting key 31f94ea2efc500b757d55f875cb87497
Aug 08 03:51:51 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161111, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM
I20260808 03:51:51.475687 3997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:51.457375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:47793 (local address 127.26.146.190:37325)
0808 03:51:51.457824 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:51.457831 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:51.458697 (+ 866us) server_negotiation.cc:407] Connection header received
0808 03:51:51.460879 (+ 2182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:51.460884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:51.460961 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:51.461053 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:51.462908 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:51.463734 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:51.465888 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:51.466281 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:51.466373 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:51.470292 (+ 3919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:51.470298 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:51.470301 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:51.470332 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:51.472786 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:51.473464 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:51.473469 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:51.473473 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:51.473620 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:51.474013 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:51.474017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:51.474020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:51.474317 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:51.474431 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:51.475092 (+ 661us) server_negotiation.cc:299] Negotiation successful
0808 03:51:51.475317 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":216,"threads_started":1}
I20260808 03:51:51.476373 3999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:51.457858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.129:47793)
0808 03:51:51.458540 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:51.458579 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:51.460561 (+ 1982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:51.461385 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:51.461396 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:51.462076 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:51.462728 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:51.462752 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:51.463931 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:51.463936 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:51.465762 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:51.465777 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:51.466493 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:51.466500 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:51.466783 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:51.467464 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:51.467493 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:51.470061 (+ 2568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:51.472932 (+ 2871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:51.472941 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:51.472953 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:51.473352 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:51.473713 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:51.473716 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:51.473719 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:51.473905 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:51.474409 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:51.474414 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:51.474716 (+ 302us) client_negotiation.cc:769] Sending connection context
0808 03:51:51.474926 (+ 210us) client_negotiation.cc:240] Negotiation successful
0808 03:51:51.475303 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":251,"threads_started":1}
I20260808 03:51:51.477885 3995 heartbeater.cc:347] Connected to a master server at 127.26.146.190:37325
I20260808 03:51:51.478173 3995 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:51.478890 3995 heartbeater.cc:511] Master 127.26.146.190:37325 requested a full tablet report, sending...
I20260808 03:51:51.480970 3720 ts_manager.cc:194] Registered new tserver with Master: 502e92ad9ce04a7ba76a5e857a7e51d6 (127.26.146.129:38971)
I20260808 03:51:51.482556 3720 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.146.129@KRBTEST.COM'} at 127.26.146.129:47793
WARNING: no policy specified for oryx/127.26.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.130.keytab.
Entry for principal oryx/127.26.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.130.keytab.
I20260808 03:51:51.515300 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:37325
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.130.keytab
--principal=oryx/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:40669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:51.842831 4003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:51.843127 4003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:51.843259 4003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:51.850431 4003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:51.850561 4003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:51.850719 4003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:51:51.860401 4003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:40669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.130.keytab
--principal=oryx/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:37325
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:51.861694 4003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:51.863039 4003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:51.876138 4012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:51.876116 4013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:51.876077 4020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:51.876756 4003 server_base.cc:1034] running on GCE node
I20260808 03:51:51.877339 4003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:51.878213 4003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:51.879416 4003 hybrid_clock.cc:648] HybridClock initialized: now 1786161111879370 us; error 31 us; skew 500 ppm
I20260808 03:51:51.879702 4003 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:51 dist-test-slave-jlzn krb5kdc[3656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161111, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:51.882723 4003 init.cc:377] Logged in from keytab as oryx/127.26.146.130@KRBTEST.COM (short username oryx)
I20260808 03:51:51.884158 4003 webserver.cc:460] Webserver started at http://127.26.146.130:33401/ using document root <none> and password file <none>
I20260808 03:51:51.884871 4003 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:51.884979 4003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:51.885236 4003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:51.888039 4003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "d330735cea094acfaa5238dab0764ce1"
format_stamp: "Formatted at 2026-08-08 03:51:51 on dist-test-slave-jlzn"
server_key: "c15399655391deb017ef4cbbb0a8e956"
server_key_iv: "eb37299bcd93770571db1b23f510c994"
server_key_version: "encryptionkey@0"
I20260808 03:51:51.888790 4003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "d330735cea094acfaa5238dab0764ce1"
format_stamp: "Formatted at 2026-08-08 03:51:51 on dist-test-slave-jlzn"
server_key: "c15399655391deb017ef4cbbb0a8e956"
server_key_iv: "eb37299bcd93770571db1b23f510c994"
server_key_version: "encryptionkey@0"
I20260808 03:51:51.893639 4003 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:51:51.898229 4033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:51.899500 4003 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260808 03:51:51.899650 4003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "d330735cea094acfaa5238dab0764ce1"
format_stamp: "Formatted at 2026-08-08 03:51:51 on dist-test-slave-jlzn"
server_key: "c15399655391deb017ef4cbbb0a8e956"
server_key_iv: "eb37299bcd93770571db1b23f510c994"
server_key_version: "encryptionkey@0"
I20260808 03:51:51.899854 4003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:51.921005 4003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:51.925275 4003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:51.925562 4003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:51.926527 4003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:51.927860 4003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:51.927945 4003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:51.928066 4003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:51.928122 4003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:51.956997 4003 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:38737
I20260808 03:51:51.957031 4189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:38737 every 8 connection(s)
I20260808 03:51:51.958267 4003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:51:51.960003 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 4003
I20260808 03:51:51.960171 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:51:51.960780 27210 external_mini_cluster.cc:1428] Setting key eb79b34f79bbf49a3dc566919a82c37c
Aug 08 03:51:51 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161111, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM
I20260808 03:51:51.978389 3997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:51.960882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:53741 (local address 127.26.146.190:37325)
0808 03:51:51.961048 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:51.961053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:51.962385 (+ 1332us) server_negotiation.cc:407] Connection header received
0808 03:51:51.964303 (+ 1918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:51.964308 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:51.964393 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:51.964474 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:51.965881 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:51.966564 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:51.968086 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:51.968507 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:51.968615 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:51.971965 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:51.971971 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:51.971974 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:51.972015 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:51.975145 (+ 3130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:51.976097 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:51.976102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:51.976105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:51.976246 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:51.976673 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:51.976678 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:51.976681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:51.977051 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:51.977211 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:51.977824 (+ 613us) server_negotiation.cc:299] Negotiation successful
0808 03:51:51.978069 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 03:51:51.979158 4196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:51.961486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.130:53741)
0808 03:51:51.962237 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:51.962288 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:51.964054 (+ 1766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:51.964698 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:51.964709 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:51.965181 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:51.965727 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:51.965744 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:51.966705 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:51.966709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:51.967954 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:51.967967 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:51.968714 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:51.968722 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:51.968935 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:51.969552 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:51.969574 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:51.971726 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:51.975314 (+ 3588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:51.975323 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:51.975340 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:51.975957 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:51.976369 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:51.976372 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:51.976375 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:51.976559 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:51.977165 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:51.977171 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:51.977471 (+ 300us) client_negotiation.cc:769] Sending connection context
0808 03:51:51.977683 (+ 212us) client_negotiation.cc:240] Negotiation successful
0808 03:51:51.978065 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":344,"threads_started":1}
I20260808 03:51:51.980546 4190 heartbeater.cc:347] Connected to a master server at 127.26.146.190:37325
I20260808 03:51:51.980830 4190 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:51.981632 4190 heartbeater.cc:511] Master 127.26.146.190:37325 requested a full tablet report, sending...
I20260808 03:51:51.983050 3720 ts_manager.cc:194] Registered new tserver with Master: d330735cea094acfaa5238dab0764ce1 (127.26.146.130:38737)
I20260808 03:51:51.983903 3720 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.146.130@KRBTEST.COM'} at 127.26.146.130:53741
WARNING: no policy specified for oryx/127.26.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.131.keytab.
Entry for principal oryx/127.26.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.131.keytab.
I20260808 03:51:52.014770 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:37325
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.131.keytab
--principal=oryx/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:40669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:52.338160 4201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:52.338423 4201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:52.338567 4201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:52.345662 4201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:52.345770 4201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:52.345919 4201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:51:52.355579 4201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:40669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.131.keytab
--principal=oryx/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:37325
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:52.356791 4201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:52.358070 4201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:52.370738 4209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:52.370692 4210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:52.370772 4214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:52.371793 4201 server_base.cc:1034] running on GCE node
I20260808 03:51:52.372434 4201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:52.373334 4201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:52.374603 4201 hybrid_clock.cc:648] HybridClock initialized: now 1786161112374482 us; error 46 us; skew 500 ppm
I20260808 03:51:52.374893 4201 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:52 dist-test-slave-jlzn krb5kdc[3656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161112, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:52.378055 4201 init.cc:377] Logged in from keytab as oryx/127.26.146.131@KRBTEST.COM (short username oryx)
I20260808 03:51:52.379426 4201 webserver.cc:460] Webserver started at http://127.26.146.131:43961/ using document root <none> and password file <none>
I20260808 03:51:52.380144 4201 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:52.380255 4201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:52.380602 4201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:52.383433 4201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "d02e63de991343c891c7a1054e10471d"
format_stamp: "Formatted at 2026-08-08 03:51:52 on dist-test-slave-jlzn"
server_key: "e43f8d999716641d7e7af41513462028"
server_key_iv: "0ca2cb150eb43cac71caf914ae1560dc"
server_key_version: "encryptionkey@0"
I20260808 03:51:52.384202 4201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "d02e63de991343c891c7a1054e10471d"
format_stamp: "Formatted at 2026-08-08 03:51:52 on dist-test-slave-jlzn"
server_key: "e43f8d999716641d7e7af41513462028"
server_key_iv: "0ca2cb150eb43cac71caf914ae1560dc"
server_key_version: "encryptionkey@0"
I20260808 03:51:52.389149 4201 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 03:51:52.392412 4226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:52.393668 4201 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:51:52.393796 4201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "d02e63de991343c891c7a1054e10471d"
format_stamp: "Formatted at 2026-08-08 03:51:52 on dist-test-slave-jlzn"
server_key: "e43f8d999716641d7e7af41513462028"
server_key_iv: "0ca2cb150eb43cac71caf914ae1560dc"
server_key_version: "encryptionkey@0"
I20260808 03:51:52.393958 4201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:52.423208 4201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:52.427479 4201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:52.427733 4201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:52.428678 4201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:52.430027 4201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:52.430100 4201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:52.430190 4201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:52.430233 4201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:52.461000 4201 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:37663
I20260808 03:51:52.461038 4393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:37663 every 8 connection(s)
I20260808 03:51:52.462205 4201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:51:52.466096 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 4201
I20260808 03:51:52.466281 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:51:52.466871 27210 external_mini_cluster.cc:1428] Setting key ce15a7b3bd3c4e375450de3f396c0a02
Aug 08 03:51:52 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161112, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM
I20260808 03:51:52.481916 3997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:52.464654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:50085 (local address 127.26.146.190:37325)
0808 03:51:52.464864 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:52.464871 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:52.466088 (+ 1217us) server_negotiation.cc:407] Connection header received
0808 03:51:52.468347 (+ 2259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:52.468352 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:52.468423 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:52.468501 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:52.470170 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:52.470910 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:52.472590 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:52.473002 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:52.473110 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:52.476518 (+ 3408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:52.476523 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:52.476526 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:52.476558 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:52.478823 (+ 2265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:52.479453 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:52.479457 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:52.479459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:52.479580 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:52.480060 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:52.480066 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:52.480069 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:52.480444 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:52.480590 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:52.481398 (+ 808us) server_negotiation.cc:299] Negotiation successful
0808 03:51:52.481610 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 03:51:52.482837 4399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:52.465162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.131:50085)
0808 03:51:52.465897 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:52.465954 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:52.468077 (+ 2123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:52.468749 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:52.468757 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:52.469333 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:52.470018 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:52.470043 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:52.471043 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:52.471049 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:52.472462 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:52.472474 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:52.473193 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:52.473198 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:52.473389 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:52.473990 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:52.474012 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:52.476331 (+ 2319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:52.478942 (+ 2611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:52.478949 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:52.478960 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:52.479362 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:52.479686 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:52.479689 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:52.479692 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:52.479939 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:52.480586 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:52.480592 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:52.480972 (+ 380us) client_negotiation.cc:769] Sending connection context
0808 03:51:52.481248 (+ 276us) client_negotiation.cc:240] Negotiation successful
0808 03:51:52.481674 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":256,"threads_started":1}
I20260808 03:51:52.483975 4394 heartbeater.cc:347] Connected to a master server at 127.26.146.190:37325
I20260808 03:51:52.484265 4394 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:52.484958 4394 heartbeater.cc:511] Master 127.26.146.190:37325 requested a full tablet report, sending...
I20260808 03:51:52.485469 3995 heartbeater.cc:503] Master 127.26.146.190:37325 was elected leader, sending a full tablet report...
I20260808 03:51:52.486510 3719 ts_manager.cc:194] Registered new tserver with Master: d02e63de991343c891c7a1054e10471d (127.26.146.131:37663)
I20260808 03:51:52.487365 3719 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.146.131@KRBTEST.COM'} at 127.26.146.131:50085
I20260808 03:51:52.494355 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.26.146.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.146.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.146.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.189.keytab.
Entry for principal oryx/127.26.146.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.189.keytab.
I20260808 03:51:52.545692 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.189:41585
--webserver_interface=127.26.146.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.189.keytab
--principal=oryx/127.26.146.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:40669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:37325,127.26.146.189:41585
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:51:52.868430 4404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:52.868911 4404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:52.869095 4404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:52.878914 4404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:51:52.879026 4404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:52.879099 4404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:52.879172 4404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:51:52.879227 4404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:51:52.889465 4404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:40669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:37325,127.26.146.189:41585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.189.keytab
--principal=oryx/127.26.146.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.189:41585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.146.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
W20260808 03:51:52.889824 4404 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.26.146.190:37325,127.26.146.189:41585'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 03:51:52.890865 4404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:52.892230 4404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:52.904066 4411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:52.904254 4414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:52.904654 4409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:52.904806 4404 server_base.cc:1034] running on GCE node
I20260808 03:51:52.905690 4404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:52.906977 4404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:52.908222 4404 hybrid_clock.cc:648] HybridClock initialized: now 1786161112908143 us; error 43 us; skew 500 ppm
I20260808 03:51:52.908536 4404 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:52 dist-test-slave-jlzn krb5kdc[3656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161112, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:52.911479 4404 init.cc:377] Logged in from keytab as oryx/127.26.146.189@KRBTEST.COM (short username oryx)
I20260808 03:51:52.912891 4404 webserver.cc:460] Webserver started at http://127.26.146.189:40273/ using document root <none> and password file <none>
I20260808 03:51:52.913543 4404 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:52.913642 4404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:52.913929 4404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:52.916872 4404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/data/instance:
uuid: "d8fb352759794eaf894af22d78c9b648"
format_stamp: "Formatted at 2026-08-08 03:51:52 on dist-test-slave-jlzn"
server_key: "9e715df887e902e9e9da79dcf4283c4e"
server_key_iv: "ef74a93b8fb3f5518c3de20c3f9fe3d0"
server_key_version: "encryptionkey@0"
I20260808 03:51:52.917603 4404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/wal/instance:
uuid: "d8fb352759794eaf894af22d78c9b648"
format_stamp: "Formatted at 2026-08-08 03:51:52 on dist-test-slave-jlzn"
server_key: "9e715df887e902e9e9da79dcf4283c4e"
server_key_iv: "ef74a93b8fb3f5518c3de20c3f9fe3d0"
server_key_version: "encryptionkey@0"
I20260808 03:51:52.922586 4404 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 03:51:52.925840 4426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:52.927060 4404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:51:52.927227 4404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/wal
uuid: "d8fb352759794eaf894af22d78c9b648"
format_stamp: "Formatted at 2026-08-08 03:51:52 on dist-test-slave-jlzn"
server_key: "9e715df887e902e9e9da79dcf4283c4e"
server_key_iv: "ef74a93b8fb3f5518c3de20c3f9fe3d0"
server_key_version: "encryptionkey@0"
I20260808 03:51:52.927420 4404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:52.949676 4404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:52.953685 4404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:52.953927 4404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:52.971696 4404 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.189:41585
I20260808 03:51:52.971729 4499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.189:41585 every 8 connection(s)
I20260808 03:51:52.973052 4404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/data/info.pb
I20260808 03:51:52.976832 4501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:52.976943 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 4404
I20260808 03:51:52.986330 4190 heartbeater.cc:503] Master 127.26.146.190:37325 was elected leader, sending a full tablet report...
I20260808 03:51:52.986735 4501 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:51:52 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.146.189@KRBTEST.COM
Aug 08 03:51:52 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161112, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.189@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM
I20260808 03:51:53.000034 4504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:52.979179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54978 (local address 127.26.146.189:41585)
0808 03:51:52.980042 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:52.980055 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:52.980105 (+ 50us) server_negotiation.cc:407] Connection header received
0808 03:51:52.981794 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:52.981817 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:52.982228 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:52.982616 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:52.984460 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:52.985711 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:52.987468 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:52.988355 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:52.988496 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:52.992994 (+ 4498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:52.993003 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:52.993013 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:52.993049 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:52.996120 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:52.996748 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:52.996754 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:52.996760 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:52.996917 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:52.997381 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:52.997386 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:52.997389 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:52.997918 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:52.998212 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:52.998671 (+ 459us) server_negotiation.cc:299] Negotiation successful
0808 03:51:52.999037 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":373,"threads_started":1}
I20260808 03:51:53.003512 4506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:52.989702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.0.0.1:34394)
0808 03:51:52.990377 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:52.990410 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:52.990548 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:52.990917 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:52.990924 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:52.991524 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:52.991754 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:52.991763 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:52.992825 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:52.992829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:52.994059 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:52.994071 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:52.994886 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:52.994893 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:52.995038 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:52.995604 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:52.995619 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:52.997804 (+ 2185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:53.001008 (+ 3204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.001016 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.001021 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.001597 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.001955 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.001959 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.001962 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.002139 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.002647 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:53.002654 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:53.002791 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 03:51:53.002964 (+ 173us) client_negotiation.cc:240] Negotiation successful
0808 03:51:53.003228 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":357,"threads_started":1}
I20260808 03:51:53.003513 4505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:52.989615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34394 (local address 127.26.146.190:37325)
0808 03:51:52.990061 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:52.990067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:52.990492 (+ 425us) server_negotiation.cc:407] Connection header received
0808 03:51:52.990665 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:52.990669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:52.990746 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:52.990834 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:52.991924 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:52.992685 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:52.994196 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:52.994690 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:52.994809 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:52.997977 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:52.997981 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:52.997985 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:52.998015 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:53.000892 (+ 2877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.001713 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.001717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.001719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.001846 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.002237 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.002240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.002242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.002514 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:53.002647 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:53.003070 (+ 423us) server_negotiation.cc:299] Negotiation successful
0808 03:51:53.003247 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":224,"threads_started":1}
I20260808 03:51:53.005199 4501 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:51:53 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161112, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.189@KRBTEST.COM for oryx/127.26.146.189@KRBTEST.COM
I20260808 03:51:53.016031 4506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.005781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:41585 (local address 127.0.0.1:54994)
0808 03:51:53.005965 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:53.005982 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:53.006068 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:53.006303 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:53.006306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:53.006589 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:53.006718 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.006725 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.007582 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.007585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.008491 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.008502 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.009109 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.009129 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.009262 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.009665 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:53.009679 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:53.011463 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:53.014123 (+ 2660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.014127 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.014129 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.014464 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.014750 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.014753 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.014755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.014864 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.015309 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:53.015312 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:53.015419 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 03:51:53.015520 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 03:51:53.015758 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 03:51:53.016242 4504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.005951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54994 (local address 127.26.146.189:41585)
0808 03:51:53.006065 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:53.006069 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:53.006083 (+ 14us) server_negotiation.cc:407] Connection header received
0808 03:51:53.006152 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:53.006155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:53.006206 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:53.006276 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:53.006825 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.007470 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.008598 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.008992 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.009081 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.011625 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:53.011630 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:53.011633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:53.011659 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:53.013995 (+ 2336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.014554 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.014558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.014560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.014666 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.014952 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.014955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.014957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.015217 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:53.015317 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:53.015590 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 03:51:53.015932 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":111,"server-negotiator.queue_time_us":29}
I20260808 03:51:53.020529 4501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648: Bootstrap starting.
I20260808 03:51:53.023439 4501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:53.024421 4501 log.cc:824] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:53.026978 4501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648: No bootstrap required, opened a new log
I20260808 03:51:53.033818 4501 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } }
I20260808 03:51:53.034108 4501 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:53.034178 4501 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8fb352759794eaf894af22d78c9b648, State: Initialized, Role: FOLLOWER
I20260808 03:51:53.034678 4501 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } }
I20260808 03:51:53.035420 4508 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } } }
I20260808 03:51:53.035753 4508 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:51:53.036392 4501 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 03:51:53.044327 4526 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 03:51:53.044438 4526 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 03:51:53.047655 4404 master_runner.cc:428] Detected that this master d8fb352759794eaf894af22d78c9b648 is joining an existing cluster
I20260808 03:51:53.047737 4404 master_runner.cc:432] Initiating AddMaster RPC to add 127.26.146.189:41585
I20260808 03:51:53.067713 4504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.052250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54996 (local address 127.26.146.189:41585)
0808 03:51:53.052416 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:53.052422 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:53.053195 (+ 773us) server_negotiation.cc:407] Connection header received
0808 03:51:53.053393 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:53.053397 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:53.053460 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:53.053518 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:53.054798 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.055793 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.057697 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.058375 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.058467 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.060512 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:53.060520 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:53.060524 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:53.060564 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:53.064757 (+ 4193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.065538 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.065543 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.065546 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.065691 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.066077 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.066085 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.066088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.066438 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:53.066579 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:53.067072 (+ 493us) server_negotiation.cc:299] Negotiation successful
0808 03:51:53.067279 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":33}
I20260808 03:51:53.068501 4505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.051292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34400 (local address 127.26.146.190:37325)
0808 03:51:53.051440 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:53.051444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:53.052097 (+ 653us) server_negotiation.cc:407] Connection header received
0808 03:51:53.052335 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:53.052340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:53.052417 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:53.052472 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:53.054088 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.055146 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.057202 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.057816 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.057903 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.061998 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:53.062005 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:53.062010 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:53.062051 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:53.065545 (+ 3494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.066253 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.066258 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.066261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.066404 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.066839 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.066844 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.066847 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.067187 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:53.067325 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:53.067931 (+ 606us) server_negotiation.cc:299] Negotiation successful
0808 03:51:53.068165 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 03:51:53.068652 4537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.052260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:41585 (local address 127.0.0.1:54996)
0808 03:51:53.053079 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:53.053126 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:53.053277 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:53.053706 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:53.053713 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:53.054342 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:53.054665 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.054677 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.056019 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.056023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.057544 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.057558 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.058735 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.058739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.058853 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.059514 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:53.059538 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:53.060336 (+ 798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:53.064899 (+ 4563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.064904 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.064907 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.065421 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.065811 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.065815 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.065819 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.065969 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.066678 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:53.066683 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:53.066821 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 03:51:53.067974 (+ 1153us) client_negotiation.cc:240] Negotiation successful
0808 03:51:53.068229 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"mutex_wait_us":21,"thread_start_us":343,"threads_started":1}
I20260808 03:51:53.069330 4534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.051348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.0.0.1:34400)
0808 03:51:53.051995 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:53.052015 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:53.052173 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:53.052671 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:53.052679 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:53.053505 (+ 826us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:53.053892 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.053907 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.055323 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.055328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.057030 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.057045 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.058199 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.058205 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.059168 (+ 963us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.060628 (+ 1460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:53.060651 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:53.061768 (+ 1117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:53.065687 (+ 3919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.065692 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.065695 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.066106 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.066549 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.066556 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.066560 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.066717 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.067409 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:53.067417 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:53.067608 (+ 191us) client_negotiation.cc:769] Sending connection context
0808 03:51:53.068740 (+ 1132us) client_negotiation.cc:240] Negotiation successful
0808 03:51:53.068973 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"mutex_wait_us":805,"spinlock_wait_cycles":22912,"thread_start_us":331,"threads_started":1}
Aug 08 03:51:53 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161110, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.190@KRBTEST.COM for oryx/127.26.146.189@KRBTEST.COM
WARNING: no policy specified for oryx/127.26.146.188@KRBTEST.COM; defaulting to no policy
I20260808 03:51:53.088277 4539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.075081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:41585 (local address 127.0.0.1:55002)
0808 03:51:53.075676 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:53.075711 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:53.075898 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:53.076374 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:53.076381 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:53.076999 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:53.077229 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.077238 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.078165 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.078169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.079136 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.079148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.079896 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.079902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.080093 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.080706 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:53.080722 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:53.082775 (+ 2053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:53.085739 (+ 2964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.085752 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.085760 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.086136 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.086493 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.086497 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.086500 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.086679 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.087180 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:53.087188 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:53.087360 (+ 172us) client_negotiation.cc:769] Sending connection context
0808 03:51:53.087531 (+ 171us) client_negotiation.cc:240] Negotiation successful
0808 03:51:53.087865 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":236,"threads_started":1}
I20260808 03:51:53.088310 4504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.075087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55002 (local address 127.26.146.189:41585)
0808 03:51:53.076052 (+ 965us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:53.076057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:53.076072 (+ 15us) server_negotiation.cc:407] Connection header received
0808 03:51:53.076140 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:53.076143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:53.076222 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:53.076307 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:53.077363 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.078035 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.079284 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.079742 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.079873 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.082939 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:53.082943 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:53.082946 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:53.082973 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:53.085620 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.086251 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.086255 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.086257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.086386 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.086775 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.086780 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.086783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.087051 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:53.087202 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:53.087647 (+ 445us) server_negotiation.cc:299] Negotiation successful
0808 03:51:53.087878 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":859}
I20260808 03:51:53.090651 3719 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.26.146.189:41585: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d8fb352759794eaf894af22d78c9b648"
member_type: NON_VOTER
last_known_addr {
host: "127.26.146.189"
port: 41585
}
attrs {
promote: true
}
}
dest_uuid: "4ae9871e1feb40edb677f7bb5a2cb131"
cas_config_opid_index: -1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.146.188@KRBTEST.COM" created.
I20260808 03:51:53.091943 3719 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: NON_VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: true } }
I20260808 03:51:53.108412 4542 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER d8fb352759794eaf894af22d78c9b648 (127.26.146.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: NON_VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: true } } }
I20260808 03:51:53.110013 4462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:53.112742 4542 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.26.146.189:41585
I20260808 03:51:53.113305 4542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: NON_VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: true } } }
I20260808 03:51:53.113534 4542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:53.113734 4404 master.cc:537] Master@127.26.146.189:41585 shutting down...
I20260808 03:51:53.116053 4462 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 1 FOLLOWER]: Refusing update from remote peer 4ae9871e1feb40edb677f7bb5a2cb131: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 03:51:53.117285 4541 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: NON_VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 03:51:53.121028 3696 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 03:51:53.129693 3696 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 -> Peer d8fb352759794eaf894af22d78c9b648 (127.26.146.189:41585): Couldn't send request to peer d8fb352759794eaf894af22d78c9b648. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 03:51:53.130086 4404 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:51:53.130491 4404 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:53.130709 4404 tablet_replica.cc:330] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.146.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.188.keytab.
Entry for principal oryx/127.26.146.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.188.keytab.
I20260808 03:51:53.133354 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.188:35251
--webserver_interface=127.26.146.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.188.keytab
--principal=oryx/127.26.146.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:40669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:37325,127.26.146.189:41585,127.26.146.188:35251
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:51:53.139066 4404 master.cc:552] Master@127.26.146.189:41585 shutdown complete.
I20260808 03:51:53.139196 4404 master_runner.cc:305] Clearing existing system tablet
I20260808 03:51:53.139667 4404 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:53.139770 4404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:53.144140 4404 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 03:51:53.145788 4550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:53.146114 4404 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:53.146380 4404 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 03:51:53.146462 4404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/wal
uuid: "d8fb352759794eaf894af22d78c9b648"
format_stamp: "Formatted at 2026-08-08 03:51:52 on dist-test-slave-jlzn"
server_key: "9e715df887e902e9e9da79dcf4283c4e"
server_key_iv: "ef74a93b8fb3f5518c3de20c3f9fe3d0"
server_key_version: "encryptionkey@0"
I20260808 03:51:53.151960 4404 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:53.158705 4404 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 03:51:53.158874 4404 log.cc:1196] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 03:51:53.159313 4404 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648: Deleting consensus metadata
I20260808 03:51:53.159581 4404 master_runner.cc:315] Copying system tablet from 127.26.146.190:37325
I20260808 03:51:53.162395 4404 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648: tablet copy: Beginning tablet copy session from remote peer at address 127.26.146.190:37325
I20260808 03:51:53.178597 4558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.163390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.0.0.1:34404)
0808 03:51:53.164217 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:53.164237 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:53.164383 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:53.164835 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:53.164841 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:53.165378 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:53.165686 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.165698 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.167182 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.167188 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.168723 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.168739 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.169664 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.169671 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.169870 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.170520 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:53.170544 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:53.171574 (+ 1030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:53.175462 (+ 3888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.175468 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.175472 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.176109 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.176558 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.176563 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.176567 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.176764 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.177418 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:53.177423 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:53.177615 (+ 192us) client_negotiation.cc:769] Sending connection context
0808 03:51:53.177756 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 03:51:53.178068 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":598,"threads_started":1}
I20260808 03:51:53.179597 4505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.163430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34404 (local address 127.26.146.190:37325)
0808 03:51:53.163624 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:53.163630 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:53.164340 (+ 710us) server_negotiation.cc:407] Connection header received
0808 03:51:53.164594 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:53.164600 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:53.164691 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:53.164790 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:53.165850 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.167022 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.168893 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.169504 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.169629 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.171845 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:53.171852 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:53.171874 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:53.171917 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:53.175296 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.176238 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.176244 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.176248 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.176425 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.176899 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.176905 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.176907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.177275 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:53.177437 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:53.178952 (+ 1515us) server_negotiation.cc:299] Negotiation successful
0808 03:51:53.179187 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 03:51:53.180483 3752 tablet_copy_service.cc:140] P 4ae9871e1feb40edb677f7bb5a2cb131: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d8fb352759794eaf894af22d78c9b648 ({username='oryx', principal='oryx/127.26.146.189@KRBTEST.COM'} at 127.0.0.1:34404)
I20260808 03:51:53.180951 3752 tablet_copy_service.cc:161] P 4ae9871e1feb40edb677f7bb5a2cb131: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d8fb352759794eaf894af22d78c9b648 at {username='oryx', principal='oryx/127.26.146.189@KRBTEST.COM'} at 127.0.0.1:34404: session id = d8fb352759794eaf894af22d78c9b648-00000000000000000000000000000000
I20260808 03:51:53.185839 3752 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 03:51:53.188740 4404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 03:51:53.195768 4404 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648: tablet copy: Starting download of 0 data blocks...
I20260808 03:51:53.196238 4404 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648: tablet copy: Starting download of 1 WAL segments...
I20260808 03:51:53.200079 4404 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 03:51:53.204650 3752 tablet_copy_service.cc:342] P 4ae9871e1feb40edb677f7bb5a2cb131: Request end of tablet copy session d8fb352759794eaf894af22d78c9b648-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.26.146.189@KRBTEST.COM'} at 127.0.0.1:34404
I20260808 03:51:53.204926 3752 tablet_copy_service.cc:433] P 4ae9871e1feb40edb677f7bb5a2cb131: ending tablet copy session d8fb352759794eaf894af22d78c9b648-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d8fb352759794eaf894af22d78c9b648
I20260808 03:51:53.217787 4404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:53.220799 4560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:53.220904 4563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:53.223841 4404 server_base.cc:1034] running on GCE node
W20260808 03:51:53.224143 4565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:53.224577 4404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:53.225291 4404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:53.226562 4404 hybrid_clock.cc:648] HybridClock initialized: now 1786161113226496 us; error 33 us; skew 500 ppm
I20260808 03:51:53.226840 4404 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:53 dist-test-slave-jlzn krb5kdc[3656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161113, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:53.229177 4404 init.cc:377] Logged in from keytab as oryx/127.26.146.189@KRBTEST.COM (short username oryx)
I20260808 03:51:53.230491 4404 webserver.cc:460] Webserver started at http://127.26.146.189:45891/ using document root <none> and password file <none>
I20260808 03:51:53.230862 4404 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:53.230950 4404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:53.234087 4404 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260808 03:51:53.236238 4576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:53.236704 4404 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 03:51:53.236855 4404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/wal
uuid: "d8fb352759794eaf894af22d78c9b648"
format_stamp: "Formatted at 2026-08-08 03:51:52 on dist-test-slave-jlzn"
server_key: "9e715df887e902e9e9da79dcf4283c4e"
server_key_iv: "ef74a93b8fb3f5518c3de20c3f9fe3d0"
server_key_version: "encryptionkey@0"
I20260808 03:51:53.237025 4404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:53.270684 4404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:53.274798 4404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:53.303915 4404 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.189:41585
I20260808 03:51:53.304041 4646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.189:41585 every 8 connection(s)
I20260808 03:51:53.305177 4404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-1/data/info.pb
I20260808 03:51:53.312834 4647 sys_catalog.cc:263] Verifying existing consensus state
I20260808 03:51:53.316920 4647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648: Bootstrap starting.
I20260808 03:51:53.336275 4647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 03:51:53.336947 4647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648: Bootstrap complete.
I20260808 03:51:53.338158 4647 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: NON_VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: true } }
I20260808 03:51:53.338434 4647 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d8fb352759794eaf894af22d78c9b648, State: Initialized, Role: LEARNER
I20260808 03:51:53.338778 4647 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: NON_VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: true } }
I20260808 03:51:53.339648 4650 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: NON_VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: true } } }
I20260808 03:51:53.340003 4647 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:51:53.340168 4650 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [sys.catalog]: This master's current role is: LEARNER
I20260808 03:51:53.351652 4664 catalog_manager.cc:1259] Loaded cluster ID: 3040dace32494f9d811b8cb7d4d13dab
I20260808 03:51:53.351768 4664 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648: loading cluster ID for follower catalog manager: success
I20260808 03:51:53.355274 4664 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648: acquiring CA information for follower catalog manager: success
I20260808 03:51:53.357808 4664 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 03:51:53.489776 4394 heartbeater.cc:503] Master 127.26.146.190:37325 was elected leader, sending a full tablet report...
W20260808 03:51:53.506141 4548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:53.506428 4548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:53.506558 4548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:53.514230 4548 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:51:53.514340 4548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:53.514417 4548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:53.514492 4548 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:51:53.514551 4548 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:51:53.524845 4548 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:40669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:37325,127.26.146.189:41585,127.26.146.188:35251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.188.keytab
--principal=oryx/127.26.146.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.188:35251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.146.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:53.526319 4548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:53.527956 4548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:53.540179 4676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:53.540305 4672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:53.540251 4674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:53.541548 4548 server_base.cc:1034] running on GCE node
I20260808 03:51:53.542313 4548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:53.543736 4548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:53.544966 4548 hybrid_clock.cc:648] HybridClock initialized: now 1786161113544916 us; error 35 us; skew 500 ppm
I20260808 03:51:53.545297 4548 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:53 dist-test-slave-jlzn krb5kdc[3656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161113, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:53.548480 4548 init.cc:377] Logged in from keytab as oryx/127.26.146.188@KRBTEST.COM (short username oryx)
I20260808 03:51:53.550024 4548 webserver.cc:460] Webserver started at http://127.26.146.188:38073/ using document root <none> and password file <none>
I20260808 03:51:53.550688 4548 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:53.550813 4548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:53.551105 4548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:51:53.554087 4548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/data/instance:
uuid: "9f6bdeb4f1c5459d87a2520c59d92e76"
format_stamp: "Formatted at 2026-08-08 03:51:53 on dist-test-slave-jlzn"
server_key: "ac1d0e8ba2e425ee7212a4feab4f937d"
server_key_iv: "ebfc29afeeae90bb02826bff9129d650"
server_key_version: "encryptionkey@0"
I20260808 03:51:53.554831 4548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/wal/instance:
uuid: "9f6bdeb4f1c5459d87a2520c59d92e76"
format_stamp: "Formatted at 2026-08-08 03:51:53 on dist-test-slave-jlzn"
server_key: "ac1d0e8ba2e425ee7212a4feab4f937d"
server_key_iv: "ebfc29afeeae90bb02826bff9129d650"
server_key_version: "encryptionkey@0"
I20260808 03:51:53.560305 4548 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 03:51:53.564018 4683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:53.565429 4548 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:51:53.565657 4548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/wal
uuid: "9f6bdeb4f1c5459d87a2520c59d92e76"
format_stamp: "Formatted at 2026-08-08 03:51:53 on dist-test-slave-jlzn"
server_key: "ac1d0e8ba2e425ee7212a4feab4f937d"
server_key_iv: "ebfc29afeeae90bb02826bff9129d650"
server_key_version: "encryptionkey@0"
I20260808 03:51:53.565884 4548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:53.593643 4548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:53.598242 4548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:53.598554 4548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:53.607172 4690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.599483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:41585 (local address 127.0.0.1:55010)
0808 03:51:53.600236 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:53.600257 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:53.600388 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:53.600892 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:53.600897 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:53.600928 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:53.601188 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.601199 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.602582 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.602586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.604118 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.604133 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.606196 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.606204 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.606417 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.606556 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 03:51:53.606667 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 03:51:53.606830 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":539,"threads_started":1}
I20260808 03:51:53.607220 4692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.599693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55010 (local address 127.26.146.189:41585)
0808 03:51:53.600567 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:53.600572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:53.600587 (+ 15us) server_negotiation.cc:407] Connection header received
0808 03:51:53.600646 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:53.600650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:53.600725 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:53.600828 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:53.601334 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.602439 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.604324 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.605727 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.605890 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.606069 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:53.606719 (+ 650us) server_negotiation.cc:299] Negotiation successful
0808 03:51:53.606866 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":633,"threads_started":1}
I20260808 03:51:53.608588 4613 raft_consensus.cc:1203] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260808 03:51:53.609252 4650 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ae9871e1feb40edb677f7bb5a2cb131. Latest consensus state: current_term: 1 leader_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: NON_VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: true } } }
I20260808 03:51:53.609519 4650 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [sys.catalog]: This master's current role is: LEARNER
I20260808 03:51:53.622196 4548 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.188:35251
I20260808 03:51:53.622282 4757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.188:35251 every 8 connection(s)
I20260808 03:51:53.623632 4548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/data/info.pb
I20260808 03:51:53.627543 4758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:53.629678 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 4548
I20260808 03:51:53.637887 4758 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:51:53 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.146.188@KRBTEST.COM
Aug 08 03:51:53 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161113, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.188@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM
I20260808 03:51:53.655874 4762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.631768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45928 (local address 127.26.146.188:35251)
0808 03:51:53.632581 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:53.632592 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:53.632632 (+ 40us) server_negotiation.cc:407] Connection header received
0808 03:51:53.635090 (+ 2458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:53.635114 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:53.635503 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:53.635995 (+ 492us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:53.637623 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.639316 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.641004 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.641856 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.642048 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.647845 (+ 5797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:53.647868 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:53.647883 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:53.647928 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:53.651534 (+ 3606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.652295 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.652302 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.652309 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.652470 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.652908 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.652911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.652914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.653445 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:53.653755 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:53.654349 (+ 594us) server_negotiation.cc:299] Negotiation successful
0808 03:51:53.654795 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":378,"threads_started":1}
I20260808 03:51:53.657083 4505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.640801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34408 (local address 127.26.146.190:37325)
0808 03:51:53.640992 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:53.640998 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:53.641786 (+ 788us) server_negotiation.cc:407] Connection header received
0808 03:51:53.642055 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:53.642061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:53.642144 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:53.642247 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:53.644024 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.644771 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.646714 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.647147 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.647261 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.651282 (+ 4021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:53.651287 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:53.651290 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:53.651323 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:53.654058 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.654984 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.654989 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.654992 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.655136 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.655531 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.655534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.655536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.655854 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:53.655992 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:53.656504 (+ 512us) server_negotiation.cc:299] Negotiation successful
0808 03:51:53.656723 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 03:51:53.657284 4763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.640418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.0.0.1:34408)
0808 03:51:53.641573 (+ 1155us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:53.641621 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:53.641911 (+ 290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:53.642362 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:53.642369 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:53.643464 (+ 1095us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:53.643872 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.643887 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.644947 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.644952 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.646576 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.646589 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.647404 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.647413 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.647538 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.648272 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:53.648290 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:53.651082 (+ 2792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:53.654181 (+ 3099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.654193 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.654200 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.654874 (+ 674us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.655241 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.655246 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.655249 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.655420 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.655972 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:53.655979 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:53.656155 (+ 176us) client_negotiation.cc:769] Sending connection context
0808 03:51:53.656371 (+ 216us) client_negotiation.cc:240] Negotiation successful
0808 03:51:53.656685 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":858,"thread_start_us":712,"threads_started":1}
I20260808 03:51:53.658816 4758 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:51:53 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161113, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.188@KRBTEST.COM for oryx/127.26.146.189@KRBTEST.COM
I20260808 03:51:53.672747 4763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.659451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:41585 (local address 127.0.0.1:55018)
0808 03:51:53.659631 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:53.659646 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:53.659723 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:53.660079 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:53.660082 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:53.660377 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:53.660618 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.660628 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.662010 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.662015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.663288 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.663301 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.664186 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.664190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.664316 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.664816 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:53.664833 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:53.667000 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:53.670288 (+ 3288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.670291 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.670294 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.670807 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.671172 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.671176 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.671178 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.671310 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.671836 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:53.671841 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:53.671960 (+ 119us) client_negotiation.cc:769] Sending connection context
0808 03:51:53.672067 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 03:51:53.672367 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 03:51:53.672794 4692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.659605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55018 (local address 127.26.146.189:41585)
0808 03:51:53.659761 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:53.659766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:53.659781 (+ 15us) server_negotiation.cc:407] Connection header received
0808 03:51:53.659893 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:53.659897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:53.659960 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:53.660037 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:53.660731 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.661850 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.663453 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.664058 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.664167 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.667184 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:53.667189 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:53.667192 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:53.667232 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:53.670184 (+ 2952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.670928 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.670932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.670934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.671059 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.671409 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.671413 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.671415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.671721 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:53.671856 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:53.672141 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 03:51:53.672459 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 03:51:53.673776 4758 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:51:53 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161113, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.188@KRBTEST.COM for oryx/127.26.146.188@KRBTEST.COM
I20260808 03:51:53.687461 4763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.674747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:35251 (local address 127.0.0.1:45934)
0808 03:51:53.674924 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:53.674941 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:53.675041 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:53.675307 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:53.675311 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:53.675734 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:53.675977 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.675990 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.676899 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.676905 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.677922 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.677933 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.678707 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.678713 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.678899 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.679380 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:53.679395 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:53.681785 (+ 2390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:53.684983 (+ 3198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.684987 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.684989 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.685401 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.685738 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.685741 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.685744 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.685880 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.686391 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:53.686394 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:53.686566 (+ 172us) client_negotiation.cc:769] Sending connection context
0808 03:51:53.686719 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 03:51:53.686966 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 03:51:53.687469 4762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.674780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45934 (local address 127.26.146.188:35251)
0808 03:51:53.674925 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:53.674931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:53.674986 (+ 55us) server_negotiation.cc:407] Connection header received
0808 03:51:53.675132 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:53.675137 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:53.675191 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:53.675270 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:53.676108 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.676787 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.678045 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.678571 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.678679 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.681968 (+ 3289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:53.681972 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:53.681975 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:53.682003 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:53.684842 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.685521 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.685526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.685528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.685651 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.685964 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.685968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.685969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.686289 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:53.686394 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:53.686805 (+ 411us) server_negotiation.cc:299] Negotiation successful
0808 03:51:53.687095 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":106,"server-negotiator.queue_time_us":41}
I20260808 03:51:53.693800 4758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76: Bootstrap starting.
I20260808 03:51:53.697857 4758 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:53.699134 4758 log.cc:824] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:53.702349 4758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76: No bootstrap required, opened a new log
I20260808 03:51:53.712484 4758 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } }
I20260808 03:51:53.712910 4758 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:53.713084 4758 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f6bdeb4f1c5459d87a2520c59d92e76, State: Initialized, Role: FOLLOWER
I20260808 03:51:53.713893 4758 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } }
I20260808 03:51:53.715241 4770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } } }
I20260808 03:51:53.715768 4770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:51:53.716341 4758 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 03:51:53.729128 4782 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 03:51:53.729475 4782 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 03:51:53.734484 4548 master_runner.cc:428] Detected that this master 9f6bdeb4f1c5459d87a2520c59d92e76 is joining an existing cluster
I20260808 03:51:53.734630 4548 master_runner.cc:432] Initiating AddMaster RPC to add 127.26.146.188:35251
I20260808 03:51:53.762880 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 3808
I20260808 03:51:53.768977 4762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.740692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45948 (local address 127.26.146.188:35251)
0808 03:51:53.740888 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:53.740893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:53.749890 (+ 8997us) server_negotiation.cc:407] Connection header received
0808 03:51:53.750489 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:53.750494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:53.750569 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:53.750662 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:53.753114 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.754362 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.756945 (+ 2583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.757611 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.757702 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.760606 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:53.760612 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:53.760616 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:53.760651 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:53.765703 (+ 5052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.766652 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.766656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.766659 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.766806 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.767215 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.767220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.767222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.767561 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:53.767696 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:53.768209 (+ 513us) server_negotiation.cc:299] Negotiation successful
0808 03:51:53.768457 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 03:51:53.769662 4505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.740534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34418 (local address 127.26.146.190:37325)
0808 03:51:53.740775 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:53.740782 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:53.742243 (+ 1461us) server_negotiation.cc:407] Connection header received
0808 03:51:53.742511 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:53.742517 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:53.742598 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:53.742674 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:53.751330 (+ 8656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.752752 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.755900 (+ 3148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.756892 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.756985 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.760734 (+ 3749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:53.760743 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:53.760748 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:53.760791 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:53.765750 (+ 4959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.766667 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.766673 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.766676 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.766886 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.767407 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.767411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.767414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.767865 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:53.767993 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:53.768793 (+ 800us) server_negotiation.cc:299] Negotiation successful
0808 03:51:53.769054 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 03:51:53.770427 4797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.741741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:35251 (local address 127.0.0.1:45948)
0808 03:51:53.749792 (+ 8051us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:53.749813 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:53.750332 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:53.750897 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:53.750901 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:53.752405 (+ 1504us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:53.752954 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.752974 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.754583 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.754587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.756769 (+ 2182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.756792 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.758074 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.758082 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.758244 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.758846 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:53.758877 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:53.760427 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:53.765846 (+ 5419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.765851 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.765855 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.766539 (+ 684us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.766935 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.766939 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.766943 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.767101 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.767764 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:53.767768 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:53.767940 (+ 172us) client_negotiation.cc:769] Sending connection context
0808 03:51:53.769231 (+ 1291us) client_negotiation.cc:240] Negotiation successful
0808 03:51:53.769554 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7722,"thread_start_us":10858,"threads_started":1}
I20260808 03:51:53.771260 4795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.740626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.0.0.1:34418)
0808 03:51:53.742125 (+ 1499us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:53.742145 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:53.742354 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:53.742982 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:53.742987 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:53.750097 (+ 7110us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:53.751134 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.751148 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.752983 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.752987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.755704 (+ 2717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.755719 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.757420 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.757432 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.757742 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.758911 (+ 1169us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:53.758929 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:53.760467 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:53.765874 (+ 5407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.765878 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.765880 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.766540 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.767016 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.767020 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.767023 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.767276 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.768074 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:53.768079 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:53.768397 (+ 318us) client_negotiation.cc:769] Sending connection context
0808 03:51:53.769916 (+ 1519us) client_negotiation.cc:240] Negotiation successful
0808 03:51:53.770403 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1338,"spinlock_wait_cycles":348416,"thread_start_us":1114,"threads_started":1}
I20260808 03:51:53.775203 4692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.743008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55034 (local address 127.26.146.189:41585)
0808 03:51:53.743211 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:53.743216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:53.754108 (+ 10892us) server_negotiation.cc:407] Connection header received
0808 03:51:53.754400 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:53.754406 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:53.754489 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:53.754561 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:53.756900 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.758079 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.760840 (+ 2761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.761841 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.761941 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.765928 (+ 3987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:53.765936 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:53.765941 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:53.765985 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:53.771430 (+ 5445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.772244 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.772249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.772252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.772408 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.773019 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.773025 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.773028 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.773445 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:53.773589 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:53.774357 (+ 768us) server_negotiation.cc:299] Negotiation successful
0808 03:51:53.774642 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 03:51:53.777223 4796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.742420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:41585 (local address 127.0.0.1:55034)
0808 03:51:53.754003 (+ 11583us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:53.754026 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:53.754240 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:53.754814 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:53.754819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:53.756025 (+ 1206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:53.756658 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.756675 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.758254 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.758260 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.760642 (+ 2382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.760657 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.762409 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.762416 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.762704 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.763898 (+ 1194us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:53.763920 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:53.765700 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:53.771626 (+ 5926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.771631 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.771635 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.772107 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.772612 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.772616 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.772619 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.772881 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.773682 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:53.773687 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:53.773999 (+ 312us) client_negotiation.cc:769] Sending connection context
0808 03:51:53.774142 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 03:51:53.776477 (+ 2335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11277,"thread_start_us":6287,"threads_started":1}
Aug 08 03:51:53 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.146.129@KRBTEST.COM
Aug 08 03:51:53 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161110, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.190@KRBTEST.COM for oryx/127.26.146.188@KRBTEST.COM
I20260808 03:51:53.797942 4690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.777244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:35251 (local address 127.0.0.1:45964)
0808 03:51:53.777590 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:53.777609 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:53.777707 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:53.779100 (+ 1393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:53.779105 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:53.779860 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:53.780022 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.780033 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.781292 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.781297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.782930 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.782947 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.784680 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.784687 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.784933 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.785756 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:53.785781 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:53.789767 (+ 3986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:53.794821 (+ 5054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.794826 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.794830 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.795375 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.795780 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.795785 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.795789 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.796021 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.796673 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:53.796678 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:53.796895 (+ 217us) client_negotiation.cc:769] Sending connection context
0808 03:51:53.797092 (+ 197us) client_negotiation.cc:240] Negotiation successful
0808 03:51:53.797398 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227}
I20260808 03:51:53.798086 4762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.777918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45964 (local address 127.26.146.188:35251)
0808 03:51:53.778573 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:53.778579 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:53.778596 (+ 17us) server_negotiation.cc:407] Connection header received
0808 03:51:53.778676 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:53.778681 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:53.778766 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:53.778856 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:53.780157 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.781138 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.783122 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.784269 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.784404 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.789977 (+ 5573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:53.789984 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:53.789989 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:53.790034 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:53.794711 (+ 4677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.795486 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.795492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.795495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.795671 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.796139 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.796144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.796147 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.796542 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:53.796697 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:53.797348 (+ 651us) server_negotiation.cc:299] Negotiation successful
0808 03:51:53.797583 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548}
I20260808 03:51:53.799726 3719 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.26.146.188:35251: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76"
member_type: NON_VOTER
last_known_addr {
host: "127.26.146.188"
port: 35251
}
attrs {
promote: true
}
}
dest_uuid: "4ae9871e1feb40edb677f7bb5a2cb131"
cas_config_opid_index: 5
I20260808 03:51:53.800144 4798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.764969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37420 (local address 127.26.146.129:38971)
0808 03:51:53.772694 (+ 7725us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:53.772745 (+ 51us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:53.772780 (+ 35us) server_negotiation.cc:407] Connection header received
0808 03:51:53.772849 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:53.772858 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:53.773170 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:53.773353 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:53.775561 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.777165 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.780417 (+ 3252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.781641 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.781778 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.789257 (+ 7479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:53.789268 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:53.789283 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:53.789349 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:53.794518 (+ 5169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.795682 (+ 1164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.795693 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.795696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.795980 (+ 284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.796490 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.796495 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.796499 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.797052 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:53.798765 (+ 1713us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:53.798985 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 03:51:53.799459 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7544,"thread_start_us":9297,"threads_started":1}
I20260808 03:51:53.800911 3719 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: NON_VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: true } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: NON_VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: true } }
I20260808 03:51:53.801916 3989 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:53.802891 4541 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9f6bdeb4f1c5459d87a2520c59d92e76 (127.26.146.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: NON_VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: true } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: NON_VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: true } } }
I20260808 03:51:53.804242 4724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:53.807717 4541 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.26.146.188:35251
I20260808 03:51:53.808902 4613 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 1 LEARNER]: Refusing update from remote peer 4ae9871e1feb40edb677f7bb5a2cb131: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 03:51:53.809432 4724 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 1 FOLLOWER]: Refusing update from remote peer 4ae9871e1feb40edb677f7bb5a2cb131: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 03:51:53.810196 4548 master.cc:537] Master@127.26.146.188:35251 shutting down...
I20260808 03:51:53.825488 4541 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: NON_VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 03:51:53.826260 4541 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: NON_VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:51:53.830137 4548 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:51:53.830509 4548 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:53.830710 4548 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76: stopping tablet replica
I20260808 03:51:53.836076 4613 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9f6bdeb4f1c5459d87a2520c59d92e76 (127.26.146.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: NON_VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: true } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: NON_VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: true } } }
I20260808 03:51:53.841043 4650 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: NON_VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: true } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: NON_VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: true } } }
I20260808 03:51:53.841362 4650 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [sys.catalog]: This master's current role is: LEARNER
I20260808 03:51:53.843933 4548 master.cc:552] Master@127.26.146.188:35251 shutdown complete.
I20260808 03:51:53.844017 4548 master_runner.cc:305] Clearing existing system tablet
I20260808 03:51:53.844411 4548 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:53.844542 4548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:53.845324 4802 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131: attempting to promote NON_VOTER d8fb352759794eaf894af22d78c9b648 to VOTER
I20260808 03:51:53.845494 4800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: NON_VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: true } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: NON_VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: true } } }
I20260808 03:51:53.846101 4800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:53.852447 4548 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.002s sys 0.002s
I20260808 03:51:53.854275 4802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: NON_VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: true } }
I20260808 03:51:53.857584 4613 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 1 LEARNER]: Refusing update from remote peer 4ae9871e1feb40edb677f7bb5a2cb131: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 03:51:53.858064 4690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.857313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:35251 (local address 127.0.0.1:45978)
0808 03:51:53.857577 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:53.857793 (+ 216us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260808 03:51:53.858244 4541 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 03:51:53.860538 4809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:53.861569 4548 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:53.861829 4548 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.000s
I20260808 03:51:53.861958 4548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/wal
uuid: "9f6bdeb4f1c5459d87a2520c59d92e76"
format_stamp: "Formatted at 2026-08-08 03:51:53 on dist-test-slave-jlzn"
server_key: "ac1d0e8ba2e425ee7212a4feab4f937d"
server_key_iv: "ebfc29afeeae90bb02826bff9129d650"
server_key_version: "encryptionkey@0"
I20260808 03:51:53.861027 4802 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d8fb352759794eaf894af22d78c9b648 (127.26.146.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: NON_VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: true } } }
I20260808 03:51:53.868314 4548 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:53.869349 4613 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d8fb352759794eaf894af22d78c9b648 (127.26.146.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: NON_VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: true } } }
I20260808 03:51:53.874939 4650 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: NON_VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: true } } }
I20260808 03:51:53.875408 4650 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:51:53.877148 4541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: NON_VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: true } } }
I20260808 03:51:53.877449 4548 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 03:51:53.877468 4541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:53.877652 4548 log.cc:1196] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 03:51:53.878000 4541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: NON_VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: true } } }
I20260808 03:51:53.878334 4541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:53.878428 4548 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76: Deleting consensus metadata
I20260808 03:51:53.878753 4548 master_runner.cc:315] Copying system tablet from 127.26.146.190:37325
I20260808 03:51:53.880589 4690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.880049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:35251 (local address 127.0.0.1:45980)
0808 03:51:53.880255 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:53.880357 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 03:51:53.880954 3700 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 -> Peer 9f6bdeb4f1c5459d87a2520c59d92e76 (127.26.146.188:35251): Couldn't send request to peer 9f6bdeb4f1c5459d87a2520c59d92e76. Status: Network error: Client connection negotiation failed: client connection to 127.26.146.188:35251: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 03:51:53.889889 4548 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76: tablet copy: Beginning tablet copy session from remote peer at address 127.26.146.190:37325
I20260808 03:51:53.912418 4817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.890792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.0.0.1:34422)
0808 03:51:53.891982 (+ 1190us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:53.892002 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:53.892198 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:53.892583 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:53.892588 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:53.893346 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:53.893836 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.893852 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.895564 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.895569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.897497 (+ 1928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:53.897514 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.898745 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.898751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:53.898995 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.901827 (+ 2832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:53.901850 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:53.903299 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:53.908633 (+ 5334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.908638 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.908642 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.909238 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.909726 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:53.909731 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:53.909734 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:53.909983 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:53.910766 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:53.910772 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:53.911044 (+ 272us) client_negotiation.cc:769] Sending connection context
0808 03:51:53.911220 (+ 176us) client_negotiation.cc:240] Negotiation successful
0808 03:51:53.911641 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":945,"thread_start_us":704,"threads_started":1}
I20260808 03:51:53.912456 4505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:53.890834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34422 (local address 127.26.146.190:37325)
0808 03:51:53.891115 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:53.891122 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:53.892110 (+ 988us) server_negotiation.cc:407] Connection header received
0808 03:51:53.892334 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:53.892340 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:53.892421 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:53.892519 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:53.893993 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.895383 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.897661 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:53.898565 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:53.898679 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:53.903608 (+ 4929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:53.903616 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:53.903621 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:53.903663 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:53.908489 (+ 4826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.909365 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.909370 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.909375 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.909597 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:53.910140 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:53.910145 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:53.910149 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:53.910614 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:53.910762 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:53.911338 (+ 576us) server_negotiation.cc:299] Negotiation successful
0808 03:51:53.911619 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156}
I20260808 03:51:53.913360 3752 tablet_copy_service.cc:140] P 4ae9871e1feb40edb677f7bb5a2cb131: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9f6bdeb4f1c5459d87a2520c59d92e76 ({username='oryx', principal='oryx/127.26.146.188@KRBTEST.COM'} at 127.0.0.1:34422)
I20260808 03:51:53.913599 3752 tablet_copy_service.cc:161] P 4ae9871e1feb40edb677f7bb5a2cb131: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9f6bdeb4f1c5459d87a2520c59d92e76 at {username='oryx', principal='oryx/127.26.146.188@KRBTEST.COM'} at 127.0.0.1:34422: session id = 9f6bdeb4f1c5459d87a2520c59d92e76-00000000000000000000000000000000
I20260808 03:51:53.918385 3752 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 03:51:53.920663 4548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 03:51:53.930058 4548 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76: tablet copy: Starting download of 0 data blocks...
I20260808 03:51:53.930438 4548 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76: tablet copy: Starting download of 1 WAL segments...
I20260808 03:51:53.933703 4548 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 03:51:53.939544 3752 tablet_copy_service.cc:342] P 4ae9871e1feb40edb677f7bb5a2cb131: Request end of tablet copy session 9f6bdeb4f1c5459d87a2520c59d92e76-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.26.146.188@KRBTEST.COM'} at 127.0.0.1:34422
I20260808 03:51:53.939760 3752 tablet_copy_service.cc:433] P 4ae9871e1feb40edb677f7bb5a2cb131: ending tablet copy session 9f6bdeb4f1c5459d87a2520c59d92e76-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9f6bdeb4f1c5459d87a2520c59d92e76
I20260808 03:51:53.952183 4548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:53.955276 4818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:53.964738 4819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:53.964928 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 3808
I20260808 03:51:53.971608 4548 server_base.cc:1034] running on GCE node
W20260808 03:51:53.978897 4821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:53.979506 4548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:53.984073 4548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:53.985312 4548 hybrid_clock.cc:648] HybridClock initialized: now 1786161113985262 us; error 31 us; skew 500 ppm
I20260808 03:51:53.985560 4548 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:53 dist-test-slave-jlzn krb5kdc[3656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161113, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:53.988131 4548 init.cc:377] Logged in from keytab as oryx/127.26.146.188@KRBTEST.COM (short username oryx)
I20260808 03:51:53.998379 4548 webserver.cc:460] Webserver started at http://127.26.146.188:42655/ using document root <none> and password file <none>
I20260808 03:51:53.998909 4548 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:53.998950 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:38971
--local_ip_for_outbound_sockets=127.26.146.129
--tserver_master_addrs=127.26.146.190:37325
--webserver_port=35721
--webserver_interface=127.26.146.129
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.129.keytab
--principal=oryx/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:40669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.26.146.190:37325,127.26.146.189:41585
--tserver_master_addrs=127.26.146.190:37325,127.26.146.189:41585,127.26.146.188:35251 with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:51:53.999131 4548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:54.002897 4548 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 03:51:54.007332 4829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:54.007766 4548 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260808 03:51:54.007915 4548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/wal
uuid: "9f6bdeb4f1c5459d87a2520c59d92e76"
format_stamp: "Formatted at 2026-08-08 03:51:53 on dist-test-slave-jlzn"
server_key: "ac1d0e8ba2e425ee7212a4feab4f937d"
server_key_iv: "ebfc29afeeae90bb02826bff9129d650"
server_key_version: "encryptionkey@0"
I20260808 03:51:54.008097 4548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:54.026546 4548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:54.030567 4548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:54.056857 4548 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.188:35251
I20260808 03:51:54.057042 4907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.188:35251 every 8 connection(s)
I20260808 03:51:54.057919 4548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/master-2/data/info.pb
I20260808 03:51:54.063196 4909 sys_catalog.cc:263] Verifying existing consensus state
I20260808 03:51:54.067106 4909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76: Bootstrap starting.
I20260808 03:51:54.085600 4909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 03:51:54.086148 4909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76: Bootstrap complete.
I20260808 03:51:54.087298 4909 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: NON_VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: true } }
I20260808 03:51:54.087661 4909 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9f6bdeb4f1c5459d87a2520c59d92e76, State: Initialized, Role: LEARNER
I20260808 03:51:54.087973 4909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: NON_VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: true } }
I20260808 03:51:54.088826 4913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: NON_VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: true } } }
I20260808 03:51:54.089138 4909 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:51:54.089351 4913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [sys.catalog]: This master's current role is: LEARNER
I20260808 03:51:54.100497 4925 catalog_manager.cc:1259] Loaded cluster ID: 3040dace32494f9d811b8cb7d4d13dab
I20260808 03:51:54.100596 4925 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76: loading cluster ID for follower catalog manager: success
I20260808 03:51:54.104118 4925 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76: acquiring CA information for follower catalog manager: success
I20260808 03:51:54.106508 4925 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 03:51:54.379565 4831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:54.379876 4831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:54.380020 4831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:54.387389 4831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:54.387493 4831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:54.387647 4831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:51:54.398701 4831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:40669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.129.keytab
--principal=oryx/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:38971
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=35721
--tserver_master_addrs=127.26.146.190:37325,127.26.146.189:41585,127.26.146.188:35251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:54.400100 4831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:54.401465 4831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:54.415692 4938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:54.415659 4939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:54.415761 4942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:54.416896 4831 server_base.cc:1034] running on GCE node
I20260808 03:51:54.417460 4831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:54.418313 4831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:54.419513 4831 hybrid_clock.cc:648] HybridClock initialized: now 1786161114419467 us; error 35 us; skew 500 ppm
I20260808 03:51:54.419858 4831 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:54 dist-test-slave-jlzn krb5kdc[3656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161114, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:54.422935 4831 init.cc:377] Logged in from keytab as oryx/127.26.146.129@KRBTEST.COM (short username oryx)
I20260808 03:51:54.424311 4831 webserver.cc:460] Webserver started at http://127.26.146.129:35721/ using document root <none> and password file <none>
I20260808 03:51:54.424934 4831 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:54.425051 4831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:54.430258 4831 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 03:51:54.432942 4954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:54.434195 4831 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 03:51:54.434340 4831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "502e92ad9ce04a7ba76a5e857a7e51d6"
format_stamp: "Formatted at 2026-08-08 03:51:51 on dist-test-slave-jlzn"
server_key: "1bd36488c5ef2a9d7dff75ad76925ebd"
server_key_iv: "ec7296d32b3a8cb64701b50bd2e05a1c"
server_key_version: "encryptionkey@0"
I20260808 03:51:54.435298 4831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:54.455881 4831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:54.459576 4831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:54.459869 4831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:54.460917 4831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:54.462523 4831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:54.462641 4831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:54.462783 4831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:54.462904 4831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260808 03:51:54.470773 4962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:54.462406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45988 (local address 127.26.146.188:35251)
0808 03:51:54.464382 (+ 1976us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:54.464389 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:54.464880 (+ 491us) server_negotiation.cc:407] Connection header received
0808 03:51:54.465078 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:54.465082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:54.465158 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:54.465257 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:54.465670 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:54.466701 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:54.468470 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:54.469578 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:54.469748 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:54.469941 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:54.470370 (+ 429us) server_negotiation.cc:299] Negotiation successful
0808 03:51:54.470512 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1808,"thread_start_us":1670,"threads_started":1}
I20260808 03:51:54.470867 4961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:54.462067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:35251 (local address 127.0.0.1:45988)
0808 03:51:54.464771 (+ 2704us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:54.464788 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:54.464928 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:54.465298 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:54.465303 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:54.465331 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:54.465523 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:54.465533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:54.466855 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:54.466861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:54.468208 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:54.468236 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:54.469947 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:54.469954 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:54.470161 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:54.470211 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 03:51:54.470325 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 03:51:54.470497 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2585,"thread_start_us":2498,"threads_started":1}
I20260808 03:51:54.471947 4876 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 03:51:54.472630 4913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ae9871e1feb40edb677f7bb5a2cb131. Latest consensus state: current_term: 1 leader_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: NON_VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: true } } }
I20260808 03:51:54.472949 4913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [sys.catalog]: This master's current role is: LEARNER
I20260808 03:51:54.499591 4831 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:38971
I20260808 03:51:54.499681 5117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:38971 every 8 connection(s)
I20260808 03:51:54.501003 4831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:51:54.504400 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 4831
I20260808 03:51:54.504787 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 4003
Aug 08 03:51:54 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161114, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.188@KRBTEST.COM
I20260808 03:51:54.529942 4962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:54.504920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:44409 (local address 127.26.146.188:35251)
0808 03:51:54.505339 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:54.505346 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:54.506461 (+ 1115us) server_negotiation.cc:407] Connection header received
0808 03:51:54.509437 (+ 2976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:54.509445 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:54.509543 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:54.509651 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:54.512115 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:54.513342 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:54.516016 (+ 2674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:54.516714 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:54.516812 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:54.522301 (+ 5489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:54.522309 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:54.522315 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:54.522360 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:54.526017 (+ 3657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:54.527083 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:54.527089 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:54.527093 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:54.527241 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:54.527902 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:54.527907 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:54.527911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:54.528284 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:54.528415 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:54.529356 (+ 941us) server_negotiation.cc:299] Negotiation successful
0808 03:51:54.529590 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288}
Aug 08 03:51:54 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.146.130@KRBTEST.COM
I20260808 03:51:54.531420 5125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:54.504439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:35251 (local address 127.26.146.129:44409)
0808 03:51:54.505939 (+ 1500us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:54.505999 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:54.509104 (+ 3105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:54.510078 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:54.510090 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:54.510925 (+ 835us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:54.511888 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:54.511923 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:54.513555 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:54.513561 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:54.515836 (+ 2275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:54.515853 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:54.517181 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:54.517188 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:54.517514 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:54.518497 (+ 983us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:54.518529 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:54.521991 (+ 3462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:54.526229 (+ 4238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:54.526240 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:54.526259 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:54.526915 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:54.527386 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:54.527390 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:54.527394 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:54.527729 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:54.528494 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:54.528505 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:54.528897 (+ 392us) client_negotiation.cc:769] Sending connection context
0808 03:51:54.529268 (+ 371us) client_negotiation.cc:240] Negotiation successful
0808 03:51:54.529800 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1195,"spinlock_wait_cycles":19200,"thread_start_us":926,"threads_started":1}
Aug 08 03:51:54 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161114, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.189@KRBTEST.COM
I20260808 03:51:54.533403 5118 heartbeater.cc:347] Connected to a master server at 127.26.146.188:35251
I20260808 03:51:54.533757 5118 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:54.534773 5118 heartbeater.cc:511] Master 127.26.146.188:35251 requested a full tablet report, sending...
Aug 08 03:51:54 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161114, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM
I20260808 03:51:54.537992 4857 ts_manager.cc:194] Registered new tserver with Master: 502e92ad9ce04a7ba76a5e857a7e51d6 (127.26.146.129:38971)
I20260808 03:51:54.539736 5124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:54.503842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:43673 (local address 127.26.146.189:41585)
0808 03:51:54.505340 (+ 1498us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:54.505346 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:54.511940 (+ 6594us) server_negotiation.cc:407] Connection header received
0808 03:51:54.516641 (+ 4701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:54.516647 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:54.516752 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:54.516959 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:54.519649 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:54.520723 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:54.524512 (+ 3789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:54.525150 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:54.525240 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:54.533429 (+ 8189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:54.533436 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:54.533441 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:54.533482 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:54.536702 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:54.537604 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:54.537610 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:54.537615 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:54.537761 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:54.538201 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:54.538206 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:54.538209 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:54.538596 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:54.538723 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:54.539258 (+ 535us) server_negotiation.cc:299] Negotiation successful
0808 03:51:54.539457 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1370,"thread_start_us":1240,"threads_started":1}
I20260808 03:51:54.540591 5126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:54.504439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:41585 (local address 127.26.146.129:43673)
0808 03:51:54.509962 (+ 5523us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:54.509981 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:54.516327 (+ 6346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:54.517290 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:54.517295 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:54.517952 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:54.518276 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:54.518290 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:54.520962 (+ 2672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:54.520966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:54.522561 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:54.522574 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:54.525575 (+ 3001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:54.525581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:54.525764 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:54.527629 (+ 1865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:54.527651 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:54.533193 (+ 5542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:54.536861 (+ 3668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:54.536866 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:54.536869 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:54.537471 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:54.537905 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:54.537910 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:54.537916 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:54.538058 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:54.538803 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:54.538808 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:54.538962 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 03:51:54.540009 (+ 1047us) client_negotiation.cc:240] Negotiation successful
0808 03:51:54.540296 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5351,"mutex_wait_us":366,"thread_start_us":4808,"threads_started":1}
I20260808 03:51:54.540954 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:54.506235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41220 (local address 127.26.146.130:38737)
0808 03:51:54.517105 (+ 10870us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:54.517114 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:54.517144 (+ 30us) server_negotiation.cc:407] Connection header received
0808 03:51:54.517214 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:54.517222 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:54.517462 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:54.520477 (+ 3015us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:54.520718 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:54.522421 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:54.525032 (+ 2611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:54.525875 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:54.527308 (+ 1433us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:54.532266 (+ 4958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:54.532280 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:54.532296 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:54.532341 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:54.536683 (+ 4342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:54.537657 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:54.537665 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:54.537668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:54.537835 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:54.538301 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:54.538306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:54.538309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:54.538794 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:54.540103 (+ 1309us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:54.540273 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 03:51:54.540555 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10652,"thread_start_us":11356,"threads_started":1}
I20260808 03:51:54.541740 5120 heartbeater.cc:347] Connected to a master server at 127.26.146.189:41585
I20260808 03:51:54.541872 5120 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:54.542246 5120 heartbeater.cc:511] Master 127.26.146.189:41585 requested a full tablet report, sending...
I20260808 03:51:54.542264 4182 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:54.543987 4596 ts_manager.cc:194] Registered new tserver with Master: 502e92ad9ce04a7ba76a5e857a7e51d6 (127.26.146.129:38971)
I20260808 03:51:54.546012 5130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:54.510330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:60107 (local address 127.26.146.190:37325)
0808 03:51:54.526820 (+ 16490us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:54.526826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:54.527729 (+ 903us) server_negotiation.cc:407] Connection header received
0808 03:51:54.527836 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:54.527842 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:54.527927 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:54.528027 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:54.529504 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:54.530577 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:54.532835 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:54.533775 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:54.533896 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:54.538415 (+ 4519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:54.538421 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:54.538425 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:54.538462 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:54.543221 (+ 4759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:54.543830 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:54.543846 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:54.543849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:54.544002 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:54.544466 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:54.544471 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:54.544474 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:54.544838 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:54.544974 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:54.545422 (+ 448us) server_negotiation.cc:299] Negotiation successful
0808 03:51:54.545630 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16353,"thread_start_us":16493,"threads_started":1}
I20260808 03:51:54.546140 5129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:54.510300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.129:60107)
0808 03:51:54.527454 (+ 17154us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:54.527474 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:54.527608 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:54.528288 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:54.528293 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:54.528937 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:54.529309 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:54.529323 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:54.530708 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:54.530713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:54.532649 (+ 1936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:54.532664 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:54.534173 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:54.534181 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:54.534977 (+ 796us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:54.535424 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:54.535446 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:54.538102 (+ 2656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:54.543339 (+ 5237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:54.543344 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:54.543346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:54.543684 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:54.544149 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:54.544154 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:54.544158 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:54.544329 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:54.544976 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:54.545001 (+ 25us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:54.545172 (+ 171us) client_negotiation.cc:769] Sending connection context
0808 03:51:54.545303 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 03:51:54.545591 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17024,"mutex_wait_us":3,"thread_start_us":16759,"threads_started":1}
I20260808 03:51:54.547035 5119 heartbeater.cc:347] Connected to a master server at 127.26.146.190:37325
I20260808 03:51:54.547165 5119 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:54.547622 5119 heartbeater.cc:511] Master 127.26.146.190:37325 requested a full tablet report, sending...
I20260808 03:51:54.548652 3719 ts_manager.cc:194] Re-registered known tserver with Master: 502e92ad9ce04a7ba76a5e857a7e51d6 (127.26.146.129:38971)
I20260808 03:51:54.549436 3719 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.146.129@KRBTEST.COM'} at 127.26.146.129:60107
I20260808 03:51:54.627897 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 4003
I20260808 03:51:54.651518 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:38737
--local_ip_for_outbound_sockets=127.26.146.130
--tserver_master_addrs=127.26.146.190:37325
--webserver_port=33401
--webserver_interface=127.26.146.130
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.130.keytab
--principal=oryx/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:40669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.26.146.190:37325,127.26.146.189:41585
--tserver_master_addrs=127.26.146.190:37325,127.26.146.189:41585,127.26.146.188:35251 with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:51:54.894423 4802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: NON_VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: true } } }
I20260808 03:51:54.894788 4802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:54.896224 4802 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131: attempting to promote NON_VOTER 9f6bdeb4f1c5459d87a2520c59d92e76 to VOTER
I20260808 03:51:54.896795 4802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: false } }
I20260808 03:51:54.902068 4613 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 1 FOLLOWER]: Refusing update from remote peer 4ae9871e1feb40edb677f7bb5a2cb131: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260808 03:51:54.902177 4876 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 1 LEARNER]: Refusing update from remote peer 4ae9871e1feb40edb677f7bb5a2cb131: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260808 03:51:54.902781 5135 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 03:51:54.903204 5138 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 03:51:54.906158 5135 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 9f6bdeb4f1c5459d87a2520c59d92e76 (127.26.146.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: false } } }
I20260808 03:51:54.906807 4613 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 9f6bdeb4f1c5459d87a2520c59d92e76 (127.26.146.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: false } } }
I20260808 03:51:54.909046 5138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: false } } }
I20260808 03:51:54.909265 5138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:54.909520 5138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: false } } }
I20260808 03:51:54.909699 5138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:54.910727 5150 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: false } } }
I20260808 03:51:54.911113 5150 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:51:54.911623 5135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: false } } }
I20260808 03:51:54.912041 5135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:54.912299 4876 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 9f6bdeb4f1c5459d87a2520c59d92e76 (127.26.146.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: false } } }
I20260808 03:51:54.916323 4913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: false } } }
I20260808 03:51:54.916560 4913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 03:51:54.988072 5133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:54.988332 5133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:54.988430 5133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:54.995674 5133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:54.995781 5133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:54.995942 5133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:51:55.005582 5133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:40669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.130.keytab
--principal=oryx/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:38737
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=33401
--tserver_master_addrs=127.26.146.190:37325,127.26.146.189:41585,127.26.146.188:35251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:55.006810 5133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:55.008124 5133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:55.020753 5156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:55.020789 5161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:55.020829 5158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:55.021878 5133 server_base.cc:1034] running on GCE node
I20260808 03:51:55.022462 5133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:55.023399 5133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:55.024593 5133 hybrid_clock.cc:648] HybridClock initialized: now 1786161115024551 us; error 31 us; skew 500 ppm
I20260808 03:51:55.024911 5133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:55 dist-test-slave-jlzn krb5kdc[3656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161115, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:55.028028 5133 init.cc:377] Logged in from keytab as oryx/127.26.146.130@KRBTEST.COM (short username oryx)
I20260808 03:51:55.029423 5133 webserver.cc:460] Webserver started at http://127.26.146.130:33401/ using document root <none> and password file <none>
I20260808 03:51:55.030164 5133 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:55.030300 5133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:55.035538 5133 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:51:55.038229 5174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:55.039572 5133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 03:51:55.039752 5133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "d330735cea094acfaa5238dab0764ce1"
format_stamp: "Formatted at 2026-08-08 03:51:51 on dist-test-slave-jlzn"
server_key: "c15399655391deb017ef4cbbb0a8e956"
server_key_iv: "eb37299bcd93770571db1b23f510c994"
server_key_version: "encryptionkey@0"
I20260808 03:51:55.040908 5133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:55.060539 5133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:55.065492 5133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:55.065822 5133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:55.066921 5133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:55.068542 5133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:55.068639 5133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:55.068723 5133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:55.068780 5133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:55.098829 5133 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:38737
I20260808 03:51:55.098891 5331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:38737 every 8 connection(s)
I20260808 03:51:55.100286 5133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:51:55.104084 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 5133
I20260808 03:51:55.104419 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 4201
Aug 08 03:51:55 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161115, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.188@KRBTEST.COM
Aug 08 03:51:55 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161115, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM
Aug 08 03:51:55 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.146.131@KRBTEST.COM
Aug 08 03:51:55 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161115, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.189@KRBTEST.COM
I20260808 03:51:55.138621 5341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.103523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:53771 (local address 127.26.146.188:35251)
0808 03:51:55.115533 (+ 12010us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:55.115539 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:55.117248 (+ 1709us) server_negotiation.cc:407] Connection header received
0808 03:51:55.117397 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:55.117402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:55.117479 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:55.117576 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:55.121544 (+ 3968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.122664 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.124743 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.125395 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.125481 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.131510 (+ 6029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:55.131518 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:55.131523 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:55.131563 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:55.134542 (+ 2979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:55.135375 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:55.135381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:55.135385 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:55.135532 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:55.136074 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:55.136080 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:55.136084 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:55.136454 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:55.136605 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:55.137962 (+ 1357us) server_negotiation.cc:299] Negotiation successful
0808 03:51:55.138169 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11867,"thread_start_us":11711,"threads_started":1}
I20260808 03:51:55.139091 5346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.117585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:55319 (local address 127.26.146.189:41585)
0808 03:51:55.121281 (+ 3696us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:55.121287 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:55.121817 (+ 530us) server_negotiation.cc:407] Connection header received
0808 03:51:55.121895 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:55.121899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:55.121985 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:55.122113 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:55.123692 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.124759 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.126683 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.127319 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.127443 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.131404 (+ 3961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:55.131410 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:55.131414 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:55.131465 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:55.134904 (+ 3439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:55.135766 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:55.135771 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:55.135774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:55.135937 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:55.136376 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:55.136379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:55.136382 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:55.136678 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:55.136810 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:55.138557 (+ 1747us) server_negotiation.cc:299] Negotiation successful
0808 03:51:55.138767 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3568,"thread_start_us":3453,"threads_started":1}
I20260808 03:51:55.139138 5345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.105355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36364 (local address 127.26.146.131:37663)
0808 03:51:55.116037 (+ 10682us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:55.116044 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:55.116069 (+ 25us) server_negotiation.cc:407] Connection header received
0808 03:51:55.116132 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:55.116138 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:55.116360 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:55.116613 (+ 253us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:55.118822 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.120765 (+ 1943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.122776 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.123724 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.123845 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.131239 (+ 7394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:55.131254 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:55.131272 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:55.131322 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:55.135442 (+ 4120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:55.136313 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:55.136323 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:55.136326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:55.136489 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:55.137007 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:55.137011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:55.137013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:55.137405 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:55.137694 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:55.138288 (+ 594us) server_negotiation.cc:299] Negotiation successful
0808 03:51:55.138551 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10535,"thread_start_us":10365,"threads_started":1}
I20260808 03:51:55.139410 5347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.118057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:41585 (local address 127.26.146.130:55319)
0808 03:51:55.121517 (+ 3460us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:55.121576 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:55.121706 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:55.122282 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:55.122287 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:55.122994 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:55.123501 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:55.123514 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.124924 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.124929 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:55.126477 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:55.126493 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.127726 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.127732 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:55.127986 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.128567 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:55.128590 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:55.131193 (+ 2603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:55.135037 (+ 3844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:55.135042 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:55.135045 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:55.135640 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:55.136066 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:55.136071 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:55.136074 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:55.136258 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:55.136882 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:55.136886 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:55.137076 (+ 190us) client_negotiation.cc:769] Sending connection context
0808 03:51:55.137411 (+ 335us) client_negotiation.cc:240] Negotiation successful
0808 03:51:55.138008 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3343,"mutex_wait_us":29,"thread_start_us":3238,"threads_started":1}
I20260808 03:51:55.140180 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.103181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:57849 (local address 127.26.146.190:37325)
0808 03:51:55.105002 (+ 1821us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:55.105010 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:55.114000 (+ 8990us) server_negotiation.cc:407] Connection header received
0808 03:51:55.116327 (+ 2327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:55.116333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:55.116413 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:55.116534 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:55.120832 (+ 4298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.121954 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.124754 (+ 2800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.125388 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.125481 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.130561 (+ 5080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:55.130569 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:55.130573 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:55.130615 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:55.134009 (+ 3394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:55.136082 (+ 2073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:55.136088 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:55.136091 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:55.136243 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:55.136969 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:55.136973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:55.136976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:55.137294 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:55.137514 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:55.139400 (+ 1886us) server_negotiation.cc:299] Negotiation successful
0808 03:51:55.139834 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":533,"threads_started":1}
I20260808 03:51:55.140273 5343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.103684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.130:57849)
0808 03:51:55.113731 (+ 10047us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:55.113800 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:55.116051 (+ 2251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:55.117689 (+ 1638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:55.117700 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:55.120249 (+ 2549us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:55.120620 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:55.120633 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.122224 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.122229 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:55.124588 (+ 2359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:55.124599 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.125847 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.125855 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:55.126256 (+ 401us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.127087 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:55.127122 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:55.130271 (+ 3149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:55.135075 (+ 4804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:55.135080 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:55.135083 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:55.135738 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:55.136678 (+ 940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:55.136682 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:55.136685 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:55.136858 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:55.137806 (+ 948us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:55.137810 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:55.137978 (+ 168us) client_negotiation.cc:769] Sending connection context
0808 03:51:55.138107 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 03:51:55.139366 (+ 1259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9689,"mutex_wait_us":122,"spinlock_wait_cycles":3328,"thread_start_us":13091,"threads_started":1}
I20260808 03:51:55.141140 5335 heartbeater.cc:347] Connected to a master server at 127.26.146.189:41585
I20260808 03:51:55.141424 5335 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:55.141582 4388 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:55.142205 5335 heartbeater.cc:511] Master 127.26.146.189:41585 requested a full tablet report, sending...
I20260808 03:51:55.143064 5334 heartbeater.cc:347] Connected to a master server at 127.26.146.190:37325
I20260808 03:51:55.143177 5334 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:55.143560 5334 heartbeater.cc:511] Master 127.26.146.190:37325 requested a full tablet report, sending...
I20260808 03:51:55.144148 4596 ts_manager.cc:194] Registered new tserver with Master: d330735cea094acfaa5238dab0764ce1 (127.26.146.130:38737)
I20260808 03:51:55.144415 5342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.103684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:35251 (local address 127.26.146.130:53771)
0808 03:51:55.117143 (+ 13459us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:55.117161 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:55.117287 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:55.117774 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:55.117778 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:55.118673 (+ 895us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:55.119623 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:55.119648 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.122838 (+ 3190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.122843 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:55.124585 (+ 1742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:55.124599 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.125839 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.125847 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:55.126260 (+ 413us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.127164 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:55.127187 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:55.131250 (+ 4063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:55.134666 (+ 3416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:55.134675 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:55.134693 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:55.135227 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:55.135669 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:55.135673 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:55.135676 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:55.135938 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:55.136604 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:55.136612 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:55.137644 (+ 1032us) client_negotiation.cc:769] Sending connection context
0808 03:51:55.140890 (+ 3246us) client_negotiation.cc:240] Negotiation successful
0808 03:51:55.144054 (+ 3164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13329,"mutex_wait_us":174,"spinlock_wait_cycles":39808,"thread_start_us":9391,"threads_started":1}
I20260808 03:51:55.144565 3719 ts_manager.cc:194] Re-registered known tserver with Master: d330735cea094acfaa5238dab0764ce1 (127.26.146.130:38737)
I20260808 03:51:55.145771 3719 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.146.130@KRBTEST.COM'} at 127.26.146.130:57849
I20260808 03:51:55.146499 5332 heartbeater.cc:347] Connected to a master server at 127.26.146.188:35251
I20260808 03:51:55.146623 5332 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:55.146972 5332 heartbeater.cc:511] Master 127.26.146.188:35251 requested a full tablet report, sending...
I20260808 03:51:55.148058 4857 ts_manager.cc:194] Registered new tserver with Master: d330735cea094acfaa5238dab0764ce1 (127.26.146.130:38737)
I20260808 03:51:55.224718 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 4201
I20260808 03:51:55.249275 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:37663
--local_ip_for_outbound_sockets=127.26.146.131
--tserver_master_addrs=127.26.146.190:37325
--webserver_port=43961
--webserver_interface=127.26.146.131
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.131.keytab
--principal=oryx/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:40669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.26.146.190:37325,127.26.146.189:41585
--tserver_master_addrs=127.26.146.190:37325,127.26.146.189:41585,127.26.146.188:35251 with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:51:55.552074 5119 heartbeater.cc:503] Master 127.26.146.190:37325 was elected leader, sending a full tablet report...
W20260808 03:51:55.577142 5349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:51:55.577426 5349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:51:55.577574 5349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:51:55.584565 5349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:51:55.584684 5349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:51:55.584841 5349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:51:55.594362 5349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:40669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.131.keytab
--principal=oryx/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:37663
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=43961
--tserver_master_addrs=127.26.146.190:37325,127.26.146.189:41585,127.26.146.188:35251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:51:55.595647 5349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:51:55.596966 5349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:51:55.609797 5366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:55.609825 5362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:51:55.609782 5371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:51:55.610677 5349 server_base.cc:1034] running on GCE node
I20260808 03:51:55.611214 5349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:51:55.612142 5349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:51:55.613373 5349 hybrid_clock.cc:648] HybridClock initialized: now 1786161115613309 us; error 51 us; skew 500 ppm
I20260808 03:51:55.613734 5349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:51:55 dist-test-slave-jlzn krb5kdc[3656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161115, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:51:55.616835 5349 init.cc:377] Logged in from keytab as oryx/127.26.146.131@KRBTEST.COM (short username oryx)
I20260808 03:51:55.618273 5349 webserver.cc:460] Webserver started at http://127.26.146.131:43961/ using document root <none> and password file <none>
I20260808 03:51:55.618964 5349 fs_manager.cc:359] Metadata directory not provided
I20260808 03:51:55.619104 5349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:51:55.624420 5349 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260808 03:51:55.627002 5379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:55.628329 5349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 03:51:55.628494 5349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "d02e63de991343c891c7a1054e10471d"
format_stamp: "Formatted at 2026-08-08 03:51:52 on dist-test-slave-jlzn"
server_key: "e43f8d999716641d7e7af41513462028"
server_key_iv: "0ca2cb150eb43cac71caf914ae1560dc"
server_key_version: "encryptionkey@0"
I20260808 03:51:55.629441 5349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:51:55.663442 5349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:51:55.667644 5349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:51:55.667941 5349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:51:55.668895 5349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:51:55.670245 5349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:51:55.670342 5349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:55.670449 5349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:51:55.670519 5349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:51:55.700757 5349 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:37663
I20260808 03:51:55.700814 5540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:37663 every 8 connection(s)
I20260808 03:51:55.702167 5349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:51:55.712486 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 5349
Aug 08 03:51:55 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161115, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.189@KRBTEST.COM
Aug 08 03:51:55 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161115, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.188@KRBTEST.COM
Aug 08 03:51:55 dist-test-slave-jlzn krb5kdc[3656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161115, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM
I20260808 03:51:55.728922 5549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.704934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:50877 (local address 127.26.146.188:35251)
0808 03:51:55.705894 (+ 960us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:55.705900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:55.707007 (+ 1107us) server_negotiation.cc:407] Connection header received
0808 03:51:55.709655 (+ 2648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:55.709661 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:55.709751 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:55.709880 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:55.711707 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.712875 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.715765 (+ 2890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.716574 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.716659 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.721712 (+ 5053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:55.721718 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:55.721722 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:55.721762 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:55.725702 (+ 3940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:55.726535 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:55.726540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:55.726543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:55.726704 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:55.727159 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:55.727164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:55.727167 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:55.727506 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:55.727641 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:55.728327 (+ 686us) server_negotiation.cc:299] Negotiation successful
0808 03:51:55.728555 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":847,"thread_start_us":711,"threads_started":1}
I20260808 03:51:55.729300 5548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.704940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:33195 (local address 127.26.146.190:37325)
0808 03:51:55.705392 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:55.705399 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:55.707034 (+ 1635us) server_negotiation.cc:407] Connection header received
0808 03:51:55.709271 (+ 2237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:55.709277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:55.709359 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:55.709452 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:55.711672 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.712630 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.715777 (+ 3147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.716514 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.716614 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.722683 (+ 6069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:55.722687 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:55.722690 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:55.722734 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:55.726396 (+ 3662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:55.727173 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:55.727178 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:55.727181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:55.727314 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:55.727737 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:55.727742 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:55.727745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:55.728100 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:55.728230 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:55.728749 (+ 519us) server_negotiation.cc:299] Negotiation successful
0808 03:51:55.728970 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":220,"threads_started":1}
I20260808 03:51:55.730161 5553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.707199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:41585 (local address 127.26.146.131:56465)
0808 03:51:55.707759 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:55.707779 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:55.708999 (+ 1220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:55.710362 (+ 1363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:55.710366 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:55.710972 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:55.711543 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:55.711553 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.713033 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.713043 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:55.715603 (+ 2560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:55.715623 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.716896 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.716903 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:55.717442 (+ 539us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.718215 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:55.718249 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:55.721099 (+ 2850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:55.725232 (+ 4133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:55.725241 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:55.725260 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:55.725912 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:55.726375 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:55.726380 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:55.726384 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:55.726648 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:55.727411 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:55.727420 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:55.727881 (+ 461us) client_negotiation.cc:769] Sending connection context
0808 03:51:55.728206 (+ 325us) client_negotiation.cc:240] Negotiation successful
0808 03:51:55.728740 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"mutex_wait_us":523,"thread_start_us":260,"threads_started":1}
I20260808 03:51:55.730367 5551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.705589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.131:33195)
0808 03:51:55.706648 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:55.706705 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:55.708994 (+ 2289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:55.709718 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:55.709729 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:55.710601 (+ 872us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:55.711504 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:55.711526 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.712854 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.712859 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:55.715585 (+ 2726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:55.715602 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.716952 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.716967 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:55.717434 (+ 467us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.718338 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:55.718361 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:55.722519 (+ 4158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:55.726539 (+ 4020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:55.726544 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:55.726547 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:55.727033 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:55.727471 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:55.727475 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:55.727479 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:55.727617 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:55.728319 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:55.728323 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:55.728462 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 03:51:55.729551 (+ 1089us) client_negotiation.cc:240] Negotiation successful
0808 03:51:55.729821 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"mutex_wait_us":616,"spinlock_wait_cycles":2432,"thread_start_us":417,"threads_started":1}
I20260808 03:51:55.730825 5552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.705589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:35251 (local address 127.26.146.131:50877)
0808 03:51:55.706707 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:55.706725 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:55.708989 (+ 2264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:55.709917 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:55.709922 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:55.710612 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:51:55.711512 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:55.711530 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.713079 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.713083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:55.715603 (+ 2520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:55.715617 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.716986 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.716993 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:55.717428 (+ 435us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.718469 (+ 1041us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:51:55.718490 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:51:55.721500 (+ 3010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:51:55.725878 (+ 4378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:55.725883 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:55.725887 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:55.726395 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:55.726850 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:51:55.726854 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:51:55.726858 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:51:55.727033 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:51:55.727728 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:51:55.727732 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:51:55.727919 (+ 187us) client_negotiation.cc:769] Sending connection context
0808 03:51:55.729582 (+ 1663us) client_negotiation.cc:240] Negotiation successful
0808 03:51:55.729957 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":965,"mutex_wait_us":694,"spinlock_wait_cycles":15360,"thread_start_us":643,"threads_started":1}
I20260808 03:51:55.731988 5541 heartbeater.cc:347] Connected to a master server at 127.26.146.188:35251
I20260808 03:51:55.732285 5541 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:55.732275 5554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.707362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:56465 (local address 127.26.146.189:41585)
0808 03:51:55.707948 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:55.707952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:55.707969 (+ 17us) server_negotiation.cc:407] Connection header received
0808 03:51:55.710112 (+ 2143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:55.710117 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:55.710202 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:55.710311 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:55.711712 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.712860 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.715768 (+ 2908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.716454 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.716574 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.721326 (+ 4752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:55.721331 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:55.721335 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:55.721369 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:55.725055 (+ 3686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:55.726066 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:55.726072 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:55.726076 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:55.726243 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:55.726804 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:55.726810 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:55.726814 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:55.727174 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:55.727322 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:55.731035 (+ 3713us) server_negotiation.cc:299] Negotiation successful
0808 03:51:55.731278 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":346,"threads_started":1}
I20260808 03:51:55.733098 5544 heartbeater.cc:347] Connected to a master server at 127.26.146.189:41585
I20260808 03:51:55.733240 5544 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:55.733250 5541 heartbeater.cc:511] Master 127.26.146.188:35251 requested a full tablet report, sending...
I20260808 03:51:55.733300 5542 heartbeater.cc:347] Connected to a master server at 127.26.146.190:37325
I20260808 03:51:55.733433 5542 heartbeater.cc:464] Registering TS with master...
I20260808 03:51:55.733892 5544 heartbeater.cc:511] Master 127.26.146.189:41585 requested a full tablet report, sending...
I20260808 03:51:55.733975 5542 heartbeater.cc:511] Master 127.26.146.190:37325 requested a full tablet report, sending...
I20260808 03:51:55.735208 4857 ts_manager.cc:194] Registered new tserver with Master: d02e63de991343c891c7a1054e10471d (127.26.146.131:37663)
I20260808 03:51:55.736362 3719 ts_manager.cc:194] Re-registered known tserver with Master: d02e63de991343c891c7a1054e10471d (127.26.146.131:37663)
I20260808 03:51:55.737478 3719 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.146.131@KRBTEST.COM'} at 127.26.146.131:33195
I20260808 03:51:55.738202 4596 ts_manager.cc:194] Registered new tserver with Master: d02e63de991343c891c7a1054e10471d (127.26.146.131:37663)
I20260808 03:51:55.757833 5549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.740734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45990 (local address 127.26.146.188:35251)
0808 03:51:55.740909 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:55.740915 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:55.741915 (+ 1000us) server_negotiation.cc:407] Connection header received
0808 03:51:55.742189 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:55.742195 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:55.742288 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:55.742368 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:55.744516 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.745575 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.747684 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.748375 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.748467 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.752145 (+ 3678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:55.752151 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:55.752155 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:55.752193 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:55.755050 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:55.755851 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:55.755856 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:55.755860 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:55.756014 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:55.756405 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:55.756409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:55.756411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:55.756691 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:55.756947 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:55.757280 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 03:51:55.757507 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 03:51:55.759306 5548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.740577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34434 (local address 127.26.146.190:37325)
0808 03:51:55.740755 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:55.740761 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:55.741435 (+ 674us) server_negotiation.cc:407] Connection header received
0808 03:51:55.741763 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:55.741768 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:55.741846 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:55.741922 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:55.745027 (+ 3105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.745781 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.747300 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.747954 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.748047 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.751769 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:55.751776 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:55.751781 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:55.751858 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:55.755106 (+ 3248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:55.755991 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:55.755996 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:55.755999 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:55.756155 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:55.756691 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:55.756697 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:55.756700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:55.757065 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:55.757312 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:55.757632 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 03:51:55.757868 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 03:51:55.760128 5554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.741776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55046 (local address 127.26.146.189:41585)
0808 03:51:55.741954 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:55.741959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:55.744444 (+ 2485us) server_negotiation.cc:407] Connection header received
0808 03:51:55.744580 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:55.744585 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:55.744694 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:55.744827 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:55.747271 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.748260 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.750059 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.750728 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.750861 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.753691 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:55.753698 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:55.753703 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:55.753740 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:55.756974 (+ 3234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:55.757726 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:55.757731 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:55.757735 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:55.757878 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:55.758627 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:55.758632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:55.758636 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:55.758916 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:55.759100 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:55.759634 (+ 534us) server_negotiation.cc:299] Negotiation successful
0808 03:51:55.759888 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 03:51:55.763396 3719 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34434:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:51:55.770728 3719 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:51:55.798286 5576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.788985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:38971 (local address 127.0.0.1:37432)
0808 03:51:55.789727 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:55.789744 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:55.790126 (+ 382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:55.791782 (+ 1656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:55.791787 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:55.791834 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:55.792157 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:55.792167 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.793958 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.793963 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:55.796282 (+ 2319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:55.796294 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.797521 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.797527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:55.797689 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.797722 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 03:51:55.797806 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 03:51:55.797933 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":819,"threads_started":1}
I20260808 03:51:55.798322 5579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.789652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37432 (local address 127.26.146.129:38971)
0808 03:51:55.790632 (+ 980us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:55.790641 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:55.790668 (+ 27us) server_negotiation.cc:407] Connection header received
0808 03:51:55.790957 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:55.790967 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:55.791519 (+ 552us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:55.791782 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:55.792352 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.793798 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.796415 (+ 2617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.797341 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.797496 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.797682 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:55.797863 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 03:51:55.798035 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":790,"thread_start_us":682,"threads_started":1}
I20260808 03:51:55.802389 5018 tablet_service.cc:1467] Processing CreateTablet for tablet b5712cff115a4b5389e7101f011b7025 (DEFAULT_TABLE table=test-table [id=1da080cc7f5a4457bd0d68942e052e2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:55.803702 5018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5712cff115a4b5389e7101f011b7025. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:55.811626 5576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.789190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:37663 (local address 127.0.0.1:36380)
0808 03:51:55.800723 (+ 11533us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:55.800742 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:55.800850 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:55.804002 (+ 3152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:55.804008 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:55.804051 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:55.804236 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:55.804246 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.806714 (+ 2468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.806720 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:55.808635 (+ 1915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:55.808649 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.810709 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.810720 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:55.810908 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.810941 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 03:51:55.811035 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 03:51:55.811172 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11429,"thread_start_us":18844,"threads_started":1}
I20260808 03:51:55.812354 5580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.789572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36380 (local address 127.26.146.131:37663)
0808 03:51:55.802998 (+ 13426us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:55.803007 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:55.803043 (+ 36us) server_negotiation.cc:407] Connection header received
0808 03:51:55.803148 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:55.803156 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:55.803386 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:55.804417 (+ 1031us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:55.804706 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.806551 (+ 1845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.808812 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.810195 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.810362 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.810549 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:55.811873 (+ 1324us) server_negotiation.cc:299] Negotiation successful
0808 03:51:55.812040 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13225,"thread_start_us":14099,"threads_started":1}
I20260808 03:51:55.816022 5573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.787991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:38737 (local address 127.0.0.1:41224)
0808 03:51:55.789046 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:55.789074 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:55.789198 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:55.809487 (+ 20289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:55.809493 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:55.809521 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:55.809960 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:55.809973 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.811989 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.811995 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:55.813949 (+ 1954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:55.813961 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.815238 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.815244 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:55.815422 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.815456 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 03:51:55.815541 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 03:51:55.815670 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":919,"thread_start_us":836,"threads_started":1}
I20260808 03:51:55.816028 5575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.788367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41224 (local address 127.26.146.130:38737)
0808 03:51:55.808710 (+ 20343us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:55.808718 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:55.808746 (+ 28us) server_negotiation.cc:407] Connection header received
0808 03:51:55.808865 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:55.808872 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:55.809109 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:55.809560 (+ 451us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:55.810128 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.811469 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.814130 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.815037 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.815186 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.815365 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:55.815600 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 03:51:55.815765 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20166,"thread_start_us":20017,"threads_started":1}
I20260808 03:51:55.816553 5454 tablet_service.cc:1467] Processing CreateTablet for tablet b5712cff115a4b5389e7101f011b7025 (DEFAULT_TABLE table=test-table [id=1da080cc7f5a4457bd0d68942e052e2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:55.818042 5454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5712cff115a4b5389e7101f011b7025. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:55.819504 5241 tablet_service.cc:1467] Processing CreateTablet for tablet b5712cff115a4b5389e7101f011b7025 (DEFAULT_TABLE table=test-table [id=1da080cc7f5a4457bd0d68942e052e2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:55.820849 5241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5712cff115a4b5389e7101f011b7025. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:55.827950 5582 tablet_bootstrap.cc:492] T b5712cff115a4b5389e7101f011b7025 P 502e92ad9ce04a7ba76a5e857a7e51d6: Bootstrap starting.
I20260808 03:51:55.832715 5582 tablet_bootstrap.cc:654] T b5712cff115a4b5389e7101f011b7025 P 502e92ad9ce04a7ba76a5e857a7e51d6: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:55.834420 5582 log.cc:824] T b5712cff115a4b5389e7101f011b7025 P 502e92ad9ce04a7ba76a5e857a7e51d6: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:55.837358 5583 tablet_bootstrap.cc:492] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d: Bootstrap starting.
I20260808 03:51:55.841523 5582 tablet_bootstrap.cc:492] T b5712cff115a4b5389e7101f011b7025 P 502e92ad9ce04a7ba76a5e857a7e51d6: No bootstrap required, opened a new log
I20260808 03:51:55.841817 5582 ts_tablet_manager.cc:1397] T b5712cff115a4b5389e7101f011b7025 P 502e92ad9ce04a7ba76a5e857a7e51d6: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.007s
I20260808 03:51:55.843227 5583 tablet_bootstrap.cc:654] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:55.844061 5589 tablet_bootstrap.cc:492] T b5712cff115a4b5389e7101f011b7025 P d330735cea094acfaa5238dab0764ce1: Bootstrap starting.
I20260808 03:51:55.844604 5583 log.cc:824] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:55.847064 5589 tablet_bootstrap.cc:654] T b5712cff115a4b5389e7101f011b7025 P d330735cea094acfaa5238dab0764ce1: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:55.848007 5583 tablet_bootstrap.cc:492] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d: No bootstrap required, opened a new log
I20260808 03:51:55.848048 5589 log.cc:824] T b5712cff115a4b5389e7101f011b7025 P d330735cea094acfaa5238dab0764ce1: Log is configured to *not* fsync() on all Append() calls
I20260808 03:51:55.848287 5583 ts_tablet_manager.cc:1397] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260808 03:51:55.850370 5589 tablet_bootstrap.cc:492] T b5712cff115a4b5389e7101f011b7025 P d330735cea094acfaa5238dab0764ce1: No bootstrap required, opened a new log
I20260808 03:51:55.850605 5589 ts_tablet_manager.cc:1397] T b5712cff115a4b5389e7101f011b7025 P d330735cea094acfaa5238dab0764ce1: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260808 03:51:55.851888 5582 raft_consensus.cc:348] T b5712cff115a4b5389e7101f011b7025 P 502e92ad9ce04a7ba76a5e857a7e51d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d330735cea094acfaa5238dab0764ce1" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38737 } } peers { permanent_uuid: "d02e63de991343c891c7a1054e10471d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37663 } } peers { permanent_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 38971 } }
I20260808 03:51:55.852341 5582 raft_consensus.cc:374] T b5712cff115a4b5389e7101f011b7025 P 502e92ad9ce04a7ba76a5e857a7e51d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:55.852447 5582 raft_consensus.cc:729] T b5712cff115a4b5389e7101f011b7025 P 502e92ad9ce04a7ba76a5e857a7e51d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 502e92ad9ce04a7ba76a5e857a7e51d6, State: Initialized, Role: FOLLOWER
I20260808 03:51:55.853060 5582 consensus_queue.cc:260] T b5712cff115a4b5389e7101f011b7025 P 502e92ad9ce04a7ba76a5e857a7e51d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d330735cea094acfaa5238dab0764ce1" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38737 } } peers { permanent_uuid: "d02e63de991343c891c7a1054e10471d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37663 } } peers { permanent_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 38971 } }
I20260808 03:51:55.854907 5582 ts_tablet_manager.cc:1428] T b5712cff115a4b5389e7101f011b7025 P 502e92ad9ce04a7ba76a5e857a7e51d6: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260808 03:51:55.858325 5583 raft_consensus.cc:348] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d330735cea094acfaa5238dab0764ce1" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38737 } } peers { permanent_uuid: "d02e63de991343c891c7a1054e10471d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37663 } } peers { permanent_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 38971 } }
I20260808 03:51:55.858760 5583 raft_consensus.cc:374] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:55.858860 5583 raft_consensus.cc:729] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e63de991343c891c7a1054e10471d, State: Initialized, Role: FOLLOWER
I20260808 03:51:55.859520 5583 consensus_queue.cc:260] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d330735cea094acfaa5238dab0764ce1" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38737 } } peers { permanent_uuid: "d02e63de991343c891c7a1054e10471d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37663 } } peers { permanent_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 38971 } }
I20260808 03:51:55.861236 5542 heartbeater.cc:503] Master 127.26.146.190:37325 was elected leader, sending a full tablet report...
I20260808 03:51:55.863056 5589 raft_consensus.cc:348] T b5712cff115a4b5389e7101f011b7025 P d330735cea094acfaa5238dab0764ce1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d330735cea094acfaa5238dab0764ce1" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38737 } } peers { permanent_uuid: "d02e63de991343c891c7a1054e10471d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37663 } } peers { permanent_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 38971 } }
I20260808 03:51:55.863591 5589 raft_consensus.cc:374] T b5712cff115a4b5389e7101f011b7025 P d330735cea094acfaa5238dab0764ce1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:55.863731 5589 raft_consensus.cc:729] T b5712cff115a4b5389e7101f011b7025 P d330735cea094acfaa5238dab0764ce1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d330735cea094acfaa5238dab0764ce1, State: Initialized, Role: FOLLOWER
I20260808 03:51:55.864456 5589 consensus_queue.cc:260] T b5712cff115a4b5389e7101f011b7025 P d330735cea094acfaa5238dab0764ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d330735cea094acfaa5238dab0764ce1" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38737 } } peers { permanent_uuid: "d02e63de991343c891c7a1054e10471d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37663 } } peers { permanent_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 38971 } }
I20260808 03:51:55.865906 5594 raft_consensus.cc:482] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:51:55.866173 5594 raft_consensus.cc:504] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d330735cea094acfaa5238dab0764ce1" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38737 } } peers { permanent_uuid: "d02e63de991343c891c7a1054e10471d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37663 } } peers { permanent_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 38971 } }
I20260808 03:51:55.868098 5583 ts_tablet_manager.cc:1428] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d: Time spent starting tablet: real 0.020s user 0.013s sys 0.001s
I20260808 03:51:55.872581 5594 leader_election.cc:288] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d330735cea094acfaa5238dab0764ce1 (127.26.146.130:38737), 502e92ad9ce04a7ba76a5e857a7e51d6 (127.26.146.129:38971)
I20260808 03:51:55.874351 5589 ts_tablet_manager.cc:1428] T b5712cff115a4b5389e7101f011b7025 P d330735cea094acfaa5238dab0764ce1: Time spent starting tablet: real 0.024s user 0.008s sys 0.013s
I20260808 03:51:55.877164 5334 heartbeater.cc:503] Master 127.26.146.190:37325 was elected leader, sending a full tablet report...
I20260808 03:51:55.883396 5553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.868622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:38737 (local address 127.26.146.131:55053)
0808 03:51:55.869435 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:55.869454 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:55.869566 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:55.876335 (+ 6769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:55.876341 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:55.876369 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:55.876577 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:55.876591 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.879027 (+ 2436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.879033 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:55.881133 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:55.881148 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.882623 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.882629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:55.882853 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.882889 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 03:51:55.882982 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 03:51:55.883127 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673}
I20260808 03:51:55.883448 5575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.870727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:55053 (local address 127.26.146.130:38737)
0808 03:51:55.876032 (+ 5305us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:55.876037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:55.876052 (+ 15us) server_negotiation.cc:407] Connection header received
0808 03:51:55.876116 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:55.876122 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:55.876189 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:55.876289 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:55.876778 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.877891 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.881294 (+ 3403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.882453 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.882607 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.882682 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:55.883043 (+ 361us) server_negotiation.cc:299] Negotiation successful
0808 03:51:55.883179 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":5174}
I20260808 03:51:55.884279 5579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.877340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:37161 (local address 127.26.146.129:38971)
0808 03:51:55.877766 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:55.877772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:55.877791 (+ 19us) server_negotiation.cc:407] Connection header received
0808 03:51:55.877855 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:55.877860 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:55.877929 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:55.878010 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:55.878581 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.879604 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.882044 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.882963 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.883091 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.883167 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:55.883891 (+ 724us) server_negotiation.cc:299] Negotiation successful
0808 03:51:55.884022 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288}
I20260808 03:51:55.884303 5270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5712cff115a4b5389e7101f011b7025" candidate_uuid: "d02e63de991343c891c7a1054e10471d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d330735cea094acfaa5238dab0764ce1" is_pre_election: true
I20260808 03:51:55.884613 5552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.871749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:38971 (local address 127.26.146.131:37161)
0808 03:51:55.872275 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:55.872292 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:55.872411 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:55.878214 (+ 5803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:55.878219 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:55.878245 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:55.878406 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:55.878416 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.879837 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.879842 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:55.881636 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:55.881650 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.883265 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.883272 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:55.883460 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.883496 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 03:51:55.883582 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 03:51:55.883713 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":1}
I20260808 03:51:55.884886 5270 raft_consensus.cc:2454] T b5712cff115a4b5389e7101f011b7025 P d330735cea094acfaa5238dab0764ce1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e63de991343c891c7a1054e10471d in term 0.
I20260808 03:51:55.885576 5382 leader_election.cc:302] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e63de991343c891c7a1054e10471d, d330735cea094acfaa5238dab0764ce1; no voters:
I20260808 03:51:55.885968 5595 raft_consensus.cc:2781] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:51:55.885879 5047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5712cff115a4b5389e7101f011b7025" candidate_uuid: "d02e63de991343c891c7a1054e10471d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" is_pre_election: true
I20260808 03:51:55.886255 5595 raft_consensus.cc:482] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:51:55.886430 5595 raft_consensus.cc:3037] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:55.886453 5047 raft_consensus.cc:2454] T b5712cff115a4b5389e7101f011b7025 P 502e92ad9ce04a7ba76a5e857a7e51d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e63de991343c891c7a1054e10471d in term 0.
I20260808 03:51:55.889396 5595 raft_consensus.cc:504] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d330735cea094acfaa5238dab0764ce1" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38737 } } peers { permanent_uuid: "d02e63de991343c891c7a1054e10471d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37663 } } peers { permanent_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 38971 } }
I20260808 03:51:55.890393 5270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5712cff115a4b5389e7101f011b7025" candidate_uuid: "d02e63de991343c891c7a1054e10471d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d330735cea094acfaa5238dab0764ce1"
I20260808 03:51:55.890574 5270 raft_consensus.cc:3037] T b5712cff115a4b5389e7101f011b7025 P d330735cea094acfaa5238dab0764ce1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:55.893188 5270 raft_consensus.cc:2454] T b5712cff115a4b5389e7101f011b7025 P d330735cea094acfaa5238dab0764ce1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e63de991343c891c7a1054e10471d in term 1.
I20260808 03:51:55.893616 5595 leader_election.cc:288] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d [CANDIDATE]: Term 1 election: Requested vote from peers d330735cea094acfaa5238dab0764ce1 (127.26.146.130:38737), 502e92ad9ce04a7ba76a5e857a7e51d6 (127.26.146.129:38971)
I20260808 03:51:55.893653 5382 leader_election.cc:302] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e63de991343c891c7a1054e10471d, d330735cea094acfaa5238dab0764ce1; no voters:
I20260808 03:51:55.894037 5595 raft_consensus.cc:2781] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:51:55.894160 5047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5712cff115a4b5389e7101f011b7025" candidate_uuid: "d02e63de991343c891c7a1054e10471d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6"
I20260808 03:51:55.894259 5595 raft_consensus.cc:686] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d [term 1 LEADER]: Becoming Leader. State: Replica: d02e63de991343c891c7a1054e10471d, State: Running, Role: LEADER
I20260808 03:51:55.894364 5047 raft_consensus.cc:3037] T b5712cff115a4b5389e7101f011b7025 P 502e92ad9ce04a7ba76a5e857a7e51d6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:55.894796 5595 consensus_queue.cc:237] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d330735cea094acfaa5238dab0764ce1" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38737 } } peers { permanent_uuid: "d02e63de991343c891c7a1054e10471d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37663 } } peers { permanent_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 38971 } }
I20260808 03:51:55.897835 5047 raft_consensus.cc:2454] T b5712cff115a4b5389e7101f011b7025 P 502e92ad9ce04a7ba76a5e857a7e51d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e63de991343c891c7a1054e10471d in term 1.
I20260808 03:51:55.900903 3720 catalog_manager.cc:5495] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d reported cstate change: term changed from 0 to 1, leader changed from <none> to d02e63de991343c891c7a1054e10471d (127.26.146.131). New cstate: current_term: 1 leader_uuid: "d02e63de991343c891c7a1054e10471d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d330735cea094acfaa5238dab0764ce1" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e63de991343c891c7a1054e10471d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 38971 } health_report { overall_health: UNKNOWN } } }
I20260808 03:51:55.930125 5580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.925019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36384 (local address 127.26.146.131:37663)
0808 03:51:55.925217 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:55.925222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:55.925242 (+ 20us) server_negotiation.cc:407] Connection header received
0808 03:51:55.925417 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:55.925421 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:55.925509 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:55.925651 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:51:55.926004 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.926718 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.928047 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:55.928445 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:55.928557 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:55.929066 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:51:55.929555 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:51:55.929702 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:55.929784 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 03:51:55.929866 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 03:51:55.938246 5270 raft_consensus.cc:1261] T b5712cff115a4b5389e7101f011b7025 P d330735cea094acfaa5238dab0764ce1 [term 1 FOLLOWER]: Refusing update from remote peer d02e63de991343c891c7a1054e10471d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:51:55.938258 5047 raft_consensus.cc:1261] T b5712cff115a4b5389e7101f011b7025 P 502e92ad9ce04a7ba76a5e857a7e51d6 [term 1 FOLLOWER]: Refusing update from remote peer d02e63de991343c891c7a1054e10471d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:51:55.939222 5594 consensus_queue.cc:1035] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d [LEADER]: Connected to new peer: Peer: permanent_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 38971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:51:55.939519 5595 consensus_queue.cc:1035] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d330735cea094acfaa5238dab0764ce1" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:51:55.959669 5600 mvcc.cc:205] Tried to move back new op lower bound from 7316115930873106432 to 7316115930709688320. Current Snapshot: MvccSnapshot[applied={T|T < 7316115930873106432 or (T in {7316115930873106432})}]
I20260808 03:51:55.966017 5601 mvcc.cc:205] Tried to move back new op lower bound from 7316115930873106432 to 7316115930709688320. Current Snapshot: MvccSnapshot[applied={T|T < 7316115930873106432}]
I20260808 03:51:55.976605 3720 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:51:55.976917 3720 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:51:55.982781 3720 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ae9871e1feb40edb677f7bb5a2cb131: Sending DeleteTablet for 3 replicas of tablet b5712cff115a4b5389e7101f011b7025
I20260808 03:51:55.984179 5454 tablet_service.cc:1514] Processing DeleteTablet for tablet b5712cff115a4b5389e7101f011b7025 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:55 UTC) from {username='oryx', principal='oryx/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:36380
I20260808 03:51:55.984572 5018 tablet_service.cc:1514] Processing DeleteTablet for tablet b5712cff115a4b5389e7101f011b7025 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:55 UTC) from {username='oryx', principal='oryx/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:37432
I20260808 03:51:55.984987 5241 tablet_service.cc:1514] Processing DeleteTablet for tablet b5712cff115a4b5389e7101f011b7025 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:51:55 UTC) from {username='oryx', principal='oryx/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:41224
I20260808 03:51:55.996955 5620 tablet_replica.cc:330] T b5712cff115a4b5389e7101f011b7025 P d330735cea094acfaa5238dab0764ce1: stopping tablet replica
I20260808 03:51:55.997471 5620 raft_consensus.cc:2229] T b5712cff115a4b5389e7101f011b7025 P d330735cea094acfaa5238dab0764ce1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:51:55.997565 5618 tablet_replica.cc:330] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d: stopping tablet replica
I20260808 03:51:55.997916 5618 raft_consensus.cc:2229] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:51:55.998082 5620 raft_consensus.cc:2258] T b5712cff115a4b5389e7101f011b7025 P d330735cea094acfaa5238dab0764ce1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:55.998484 5618 raft_consensus.cc:2258] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:55.998900 5619 tablet_replica.cc:330] T b5712cff115a4b5389e7101f011b7025 P 502e92ad9ce04a7ba76a5e857a7e51d6: stopping tablet replica
I20260808 03:51:55.999326 5619 raft_consensus.cc:2229] T b5712cff115a4b5389e7101f011b7025 P 502e92ad9ce04a7ba76a5e857a7e51d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:51:55.999855 5619 raft_consensus.cc:2258] T b5712cff115a4b5389e7101f011b7025 P 502e92ad9ce04a7ba76a5e857a7e51d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:51:56.000991 5618 ts_tablet_manager.cc:1905] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:56.002058 5619 ts_tablet_manager.cc:1905] T b5712cff115a4b5389e7101f011b7025 P 502e92ad9ce04a7ba76a5e857a7e51d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:56.005040 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 3676
I20260808 03:51:56.006119 3762 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:51:56.007710 5620 ts_tablet_manager.cc:1905] T b5712cff115a4b5389e7101f011b7025 P d330735cea094acfaa5238dab0764ce1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:51:56.008917 5618 ts_tablet_manager.cc:1918] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:51:56.009171 5618 log.cc:1196] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/b5712cff115a4b5389e7101f011b7025
I20260808 03:51:56.009888 5618 ts_tablet_manager.cc:1939] T b5712cff115a4b5389e7101f011b7025 P d02e63de991343c891c7a1054e10471d: Deleting consensus metadata
I20260808 03:51:56.011818 3700 catalog_manager.cc:4841] TS d02e63de991343c891c7a1054e10471d (127.26.146.131:37663): tablet b5712cff115a4b5389e7101f011b7025 (table test-table [id=1da080cc7f5a4457bd0d68942e052e2d]) successfully deleted
I20260808 03:51:56.016989 5620 ts_tablet_manager.cc:1918] T b5712cff115a4b5389e7101f011b7025 P d330735cea094acfaa5238dab0764ce1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:51:56.017165 5620 log.cc:1196] T b5712cff115a4b5389e7101f011b7025 P d330735cea094acfaa5238dab0764ce1: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/b5712cff115a4b5389e7101f011b7025
I20260808 03:51:56.017586 5620 ts_tablet_manager.cc:1939] T b5712cff115a4b5389e7101f011b7025 P d330735cea094acfaa5238dab0764ce1: Deleting consensus metadata
I20260808 03:51:56.021368 5619 ts_tablet_manager.cc:1918] T b5712cff115a4b5389e7101f011b7025 P 502e92ad9ce04a7ba76a5e857a7e51d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:51:56.021543 5619 log.cc:1196] T b5712cff115a4b5389e7101f011b7025 P 502e92ad9ce04a7ba76a5e857a7e51d6: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/wal/wals/b5712cff115a4b5389e7101f011b7025
I20260808 03:51:56.022086 5619 ts_tablet_manager.cc:1939] T b5712cff115a4b5389e7101f011b7025 P 502e92ad9ce04a7ba76a5e857a7e51d6: Deleting consensus metadata
I20260808 03:51:56.023748 5554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:55.987938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55056 (local address 127.26.146.189:41585)
0808 03:51:55.988132 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:55.988139 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:56.003472 (+ 15333us) server_negotiation.cc:407] Connection header received
0808 03:51:56.004654 (+ 1182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:56.004660 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:56.004748 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:56.004822 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:56.007844 (+ 3022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:56.008971 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:56.010846 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:56.011436 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:56.011564 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:56.016025 (+ 4461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:56.016032 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:56.016038 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:56.016081 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:56.019978 (+ 3897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:56.020939 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:56.020945 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:56.020949 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:56.021109 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:56.021720 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:56.021725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:56.021729 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:56.022107 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:56.022430 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:56.023045 (+ 615us) server_negotiation.cc:299] Negotiation successful
0808 03:51:56.023331 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 03:51:56.041646 5549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:56.004060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45994 (local address 127.26.146.188:35251)
0808 03:51:56.004239 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:56.004244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:56.025230 (+ 20986us) server_negotiation.cc:407] Connection header received
0808 03:51:56.025306 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:56.025311 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:56.025395 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:56.025500 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:56.027410 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:56.028531 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:56.030341 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:56.030974 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:56.031113 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:56.034816 (+ 3703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:56.034822 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:56.034826 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:56.034864 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:56.038367 (+ 3503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:56.039167 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:56.039173 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:56.039177 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:56.039336 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:56.039772 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:56.039778 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:56.039782 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:56.040215 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:56.040488 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:56.040952 (+ 464us) server_negotiation.cc:299] Negotiation successful
0808 03:51:56.041254 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 03:51:56.156421 3698 catalog_manager.cc:4841] TS d330735cea094acfaa5238dab0764ce1 (127.26.146.130:38737): tablet b5712cff115a4b5389e7101f011b7025 (table test-table [id=1da080cc7f5a4457bd0d68942e052e2d]) successfully deleted
I20260808 03:51:56.156451 3699 catalog_manager.cc:4841] TS 502e92ad9ce04a7ba76a5e857a7e51d6 (127.26.146.129:38971): tablet b5712cff115a4b5389e7101f011b7025 (table test-table [id=1da080cc7f5a4457bd0d68942e052e2d]) successfully deleted
I20260808 03:51:56.157094 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 3676
I20260808 03:51:56.208088 5549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:56.188818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46000 (local address 127.26.146.188:35251)
0808 03:51:56.189044 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:56.189051 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:56.190582 (+ 1531us) server_negotiation.cc:407] Connection header received
0808 03:51:56.190891 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:56.190898 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:56.190991 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:56.191077 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:56.195335 (+ 4258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:56.196552 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:56.198458 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:56.199034 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:56.199148 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:56.201778 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:56.201784 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:56.201787 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:56.201822 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:56.204998 (+ 3176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:56.205836 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:56.205841 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:56.205844 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:56.206030 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:56.206437 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:56.206444 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:56.206448 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:56.206836 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:56.207093 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:56.207463 (+ 370us) server_negotiation.cc:299] Negotiation successful
0808 03:51:56.207698 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 03:51:56.208328 5554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:56.190149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55066 (local address 127.26.146.189:41585)
0808 03:51:56.190371 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:56.190378 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:56.191021 (+ 643us) server_negotiation.cc:407] Connection header received
0808 03:51:56.191304 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:56.191310 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:56.191401 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:56.191470 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:51:56.195659 (+ 4189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:56.196745 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:56.198070 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:56.198588 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:56.198703 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:56.202171 (+ 3468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:51:56.202178 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:51:56.202182 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:51:56.202223 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:51:56.205246 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:56.206038 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:56.206043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:56.206047 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:56.206194 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:51:56.206616 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:51:56.206621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:51:56.206625 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:51:56.206905 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:51:56.207147 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:56.207669 (+ 522us) server_negotiation.cc:299] Negotiation successful
0808 03:51:56.208002 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 03:51:56.958357 5632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:56.957497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.130:50533)
0808 03:51:56.957990 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:56.958143 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":238,"threads_started":1}
W20260808 03:51:56.958914 5334 heartbeater.cc:644] Failed to heartbeat to 127.26.146.190:37325 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
I20260808 03:51:56.969635 5633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:56.968665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.131:58895)
0808 03:51:56.969197 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:56.969403 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":236,"threads_started":1}
W20260808 03:51:56.970022 5542 heartbeater.cc:644] Failed to heartbeat to 127.26.146.190:37325 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
I20260808 03:51:56.976420 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:56.975649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.129:41345)
0808 03:51:56.976126 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:56.976267 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":228,"threads_started":1}
W20260808 03:51:56.976816 5119 heartbeater.cc:644] Failed to heartbeat to 127.26.146.190:37325 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
I20260808 03:51:57.482992 5638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4ae9871e1feb40edb677f7bb5a2cb131)
I20260808 03:51:57.483260 5638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: false } }
I20260808 03:51:57.484774 5638 leader_election.cc:288] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4ae9871e1feb40edb677f7bb5a2cb131 (127.26.146.190:37325), d8fb352759794eaf894af22d78c9b648 (127.26.146.189:41585)
I20260808 03:51:57.485311 5640 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4ae9871e1feb40edb677f7bb5a2cb131)
I20260808 03:51:57.485569 5640 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: false } }
I20260808 03:51:57.486758 5643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:57.484907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.0.0.1:34854)
0808 03:51:57.486260 (+ 1353us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:57.486482 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1238,"thread_start_us":1081,"threads_started":1}
I20260808 03:51:57.486965 5640 leader_election.cc:288] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4ae9871e1feb40edb677f7bb5a2cb131 (127.26.146.190:37325), 9f6bdeb4f1c5459d87a2520c59d92e76 (127.26.146.188:35251)
W20260808 03:51:57.487835 4832 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
I20260808 03:51:57.491288 5652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:57.489656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.0.0.1:34884)
0808 03:51:57.490856 (+ 1200us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:57.491056 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1095,"thread_start_us":939,"threads_started":1}
W20260808 03:51:57.491566 4577 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
I20260808 03:51:57.492673 5644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:57.491980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.0.0.1:34896)
0808 03:51:57.492252 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:57.492454 (+ 202us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
W20260808 03:51:57.493008 4832 leader_election.cc:334] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4ae9871e1feb40edb677f7bb5a2cb131 (127.26.146.190:37325): Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
I20260808 03:51:57.496013 5652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:57.495598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.0.0.1:34906)
0808 03:51:57.495771 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:57.495881 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 03:51:57.496564 5643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:57.486398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:41585 (local address 127.0.0.1:55080)
0808 03:51:57.487110 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:57.487130 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:57.487277 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:57.489983 (+ 2706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:57.489988 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:57.490016 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:57.490399 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:57.490414 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:57.491968 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:57.491973 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:57.494212 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:57.494230 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:57.495834 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:57.495840 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:57.496008 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:57.496042 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 03:51:57.496151 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 03:51:57.496290 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":1113,"threads_started":1}
I20260808 03:51:57.496686 5646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:57.487240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55080 (local address 127.26.146.189:41585)
0808 03:51:57.489702 (+ 2462us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:57.489707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:57.489725 (+ 18us) server_negotiation.cc:407] Connection header received
0808 03:51:57.489781 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:57.489786 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:57.489865 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:57.489971 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:57.490673 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:57.491784 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:57.494499 (+ 2715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:57.495662 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:57.495796 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:57.495887 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:57.496258 (+ 371us) server_negotiation.cc:299] Negotiation successful
0808 03:51:57.496362 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2334,"thread_start_us":2126,"threads_started":1}
W20260808 03:51:57.496805 4577 leader_election.cc:334] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4ae9871e1feb40edb677f7bb5a2cb131 (127.26.146.190:37325): Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
I20260808 03:51:57.497048 5647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:57.487326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:35251 (local address 127.0.0.1:46008)
0808 03:51:57.488120 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:57.488144 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:57.488262 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:57.489063 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:57.489069 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:57.489106 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:57.489541 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:57.489556 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:57.493123 (+ 3567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:57.493128 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:57.494604 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:57.494621 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:57.496114 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:57.496120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:57.496296 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:57.496327 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 03:51:57.496417 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 03:51:57.496552 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":471,"threads_started":1}
I20260808 03:51:57.497200 5648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:57.487737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46008 (local address 127.26.146.188:35251)
0808 03:51:57.488759 (+ 1022us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:57.488766 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:57.488786 (+ 20us) server_negotiation.cc:407] Connection header received
0808 03:51:57.488853 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:57.488858 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:57.488934 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:57.489039 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:57.489789 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:57.492951 (+ 3162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:57.494814 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:57.495936 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:57.496101 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:57.496164 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:57.496458 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 03:51:57.496593 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":894,"thread_start_us":763,"threads_started":1}
I20260808 03:51:57.497422 4613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d8fb352759794eaf894af22d78c9b648" is_pre_election: true
I20260808 03:51:57.497757 4613 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f6bdeb4f1c5459d87a2520c59d92e76 in term 1.
I20260808 03:51:57.497787 4876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d8fb352759794eaf894af22d78c9b648" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" is_pre_election: true
I20260808 03:51:57.498070 4876 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8fb352759794eaf894af22d78c9b648 in term 1.
I20260808 03:51:57.498221 4832 leader_election.cc:302] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9f6bdeb4f1c5459d87a2520c59d92e76, d8fb352759794eaf894af22d78c9b648; no voters: 4ae9871e1feb40edb677f7bb5a2cb131
I20260808 03:51:57.498500 4580 leader_election.cc:302] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9f6bdeb4f1c5459d87a2520c59d92e76, d8fb352759794eaf894af22d78c9b648; no voters: 4ae9871e1feb40edb677f7bb5a2cb131
I20260808 03:51:57.498560 5638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 03:51:57.498682 5638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4ae9871e1feb40edb677f7bb5a2cb131)
I20260808 03:51:57.498781 5638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 1 FOLLOWER]: Advancing to term 2
I20260808 03:51:57.498802 5640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 03:51:57.498927 5640 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4ae9871e1feb40edb677f7bb5a2cb131)
I20260808 03:51:57.499040 5640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 1 FOLLOWER]: Advancing to term 2
I20260808 03:51:57.501230 5640 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: false } }
I20260808 03:51:57.501827 5640 leader_election.cc:288] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [CANDIDATE]: Term 2 election: Requested vote from peers 4ae9871e1feb40edb677f7bb5a2cb131 (127.26.146.190:37325), 9f6bdeb4f1c5459d87a2520c59d92e76 (127.26.146.188:35251)
I20260808 03:51:57.501765 5638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: false } }
I20260808 03:51:57.502336 4876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d8fb352759794eaf894af22d78c9b648" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76"
I20260808 03:51:57.502417 5638 leader_election.cc:288] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [CANDIDATE]: Term 2 election: Requested vote from peers 4ae9871e1feb40edb677f7bb5a2cb131 (127.26.146.190:37325), d8fb352759794eaf894af22d78c9b648 (127.26.146.189:41585)
I20260808 03:51:57.502740 4876 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d8fb352759794eaf894af22d78c9b648 in current term 2: Already voted for candidate 9f6bdeb4f1c5459d87a2520c59d92e76 in this term.
I20260808 03:51:57.502811 5647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:57.502325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.0.0.1:34912)
0808 03:51:57.502594 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:57.502686 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260808 03:51:57.503499 5643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:57.503154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.0.0.1:34922)
0808 03:51:57.503304 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:57.503385 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 03:51:57.503710 4613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d8fb352759794eaf894af22d78c9b648"
I20260808 03:51:57.504006 4613 raft_consensus.cc:2379] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9f6bdeb4f1c5459d87a2520c59d92e76 in current term 2: Already voted for candidate d8fb352759794eaf894af22d78c9b648 in this term.
I20260808 03:51:57.504218 5647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:57.503722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.0.0.1:34932)
0808 03:51:57.503919 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:57.504003 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
W20260808 03:51:57.504388 4577 leader_election.cc:334] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4ae9871e1feb40edb677f7bb5a2cb131 (127.26.146.190:37325): Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
I20260808 03:51:57.504513 5643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:57.504076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.0.0.1:34948)
0808 03:51:57.504277 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:57.504378 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260808 03:51:57.504488 4577 leader_election.cc:302] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8fb352759794eaf894af22d78c9b648; no voters: 4ae9871e1feb40edb677f7bb5a2cb131, 9f6bdeb4f1c5459d87a2520c59d92e76
W20260808 03:51:57.504648 4832 leader_election.cc:334] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4ae9871e1feb40edb677f7bb5a2cb131 (127.26.146.190:37325): Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
I20260808 03:51:57.504743 5640 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 03:51:57.504784 4832 leader_election.cc:302] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9f6bdeb4f1c5459d87a2520c59d92e76; no voters: 4ae9871e1feb40edb677f7bb5a2cb131, d8fb352759794eaf894af22d78c9b648
I20260808 03:51:57.505053 5638 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 03:51:57.960393 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:57.959605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.130:48003)
0808 03:51:57.960125 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:57.960217 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":295,"threads_started":1}
I20260808 03:51:57.971441 5657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:57.970632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.131:33141)
0808 03:51:57.971154 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:57.971265 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":291,"threads_started":1}
I20260808 03:51:57.978257 5658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:57.977464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.129:42473)
0808 03:51:57.977985 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:57.978090 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":323,"threads_started":1}
I20260808 03:51:58.962289 5659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:58.961412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.130:42365)
0808 03:51:58.961968 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:58.962063 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":312,"threads_started":1}
I20260808 03:51:58.973018 5660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:58.972291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.131:60283)
0808 03:51:58.972771 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:58.972867 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":265,"threads_started":1}
I20260808 03:51:58.980259 5661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:58.979074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.129:41183)
0808 03:51:58.979866 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:58.979965 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":543,"threads_started":1}
I20260808 03:51:59.488979 5662 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:51:59.489187 5662 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: false } }
I20260808 03:51:59.489801 5662 leader_election.cc:288] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4ae9871e1feb40edb677f7bb5a2cb131 (127.26.146.190:37325), 9f6bdeb4f1c5459d87a2520c59d92e76 (127.26.146.188:35251)
I20260808 03:51:59.490370 4876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d8fb352759794eaf894af22d78c9b648" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" is_pre_election: true
I20260808 03:51:59.490605 4876 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8fb352759794eaf894af22d78c9b648 in term 2.
I20260808 03:51:59.490873 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:59.489938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.0.0.1:35250)
0808 03:51:59.490451 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:59.490628 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":270,"threads_started":1}
I20260808 03:51:59.491056 4580 leader_election.cc:302] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f6bdeb4f1c5459d87a2520c59d92e76, d8fb352759794eaf894af22d78c9b648; no voters:
I20260808 03:51:59.491384 5662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260808 03:51:59.491554 5662 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:51:59.491674 5662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 2 FOLLOWER]: Advancing to term 3
I20260808 03:51:59.492441 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:59.491999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.0.0.1:35260)
0808 03:51:59.492174 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:59.492283 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 03:51:59.492626 4577 leader_election.cc:334] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4ae9871e1feb40edb677f7bb5a2cb131 (127.26.146.190:37325): Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
I20260808 03:51:59.494076 5662 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: false } }
I20260808 03:51:59.494591 5662 leader_election.cc:288] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [CANDIDATE]: Term 3 election: Requested vote from peers 4ae9871e1feb40edb677f7bb5a2cb131 (127.26.146.190:37325), 9f6bdeb4f1c5459d87a2520c59d92e76 (127.26.146.188:35251)
I20260808 03:51:59.495039 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:59.494721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.0.0.1:35264)
0808 03:51:59.494842 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:59.494927 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 03:51:59.495065 4876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d8fb352759794eaf894af22d78c9b648" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76"
I20260808 03:51:59.495334 4876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 2 FOLLOWER]: Advancing to term 3
I20260808 03:51:59.495949 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:59.495619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.0.0.1:35280)
0808 03:51:59.495727 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:59.495788 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 03:51:59.496088 4577 leader_election.cc:334] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 4ae9871e1feb40edb677f7bb5a2cb131 (127.26.146.190:37325): Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
I20260808 03:51:59.497589 4876 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8fb352759794eaf894af22d78c9b648 in term 3.
I20260808 03:51:59.498044 4580 leader_election.cc:302] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9f6bdeb4f1c5459d87a2520c59d92e76, d8fb352759794eaf894af22d78c9b648; no voters: 4ae9871e1feb40edb677f7bb5a2cb131
I20260808 03:51:59.498340 5662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 03:51:59.498749 5662 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [term 3 LEADER]: Becoming Leader. State: Replica: d8fb352759794eaf894af22d78c9b648, State: Running, Role: LEADER
I20260808 03:51:59.499114 5662 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: false } }
I20260808 03:51:59.501231 5665 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8fb352759794eaf894af22d78c9b648. Latest consensus state: current_term: 3 leader_uuid: "d8fb352759794eaf894af22d78c9b648" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: false } } }
I20260808 03:51:59.501534 5665 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:59.502216 5668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:51:59.506266 5668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:51:59.507042 5668 catalog_manager.cc:1259] Loaded cluster ID: 3040dace32494f9d811b8cb7d4d13dab
I20260808 03:51:59.507110 5668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:51:59.508258 5668 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:51:59.509053 5668 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648: Loaded TSK: 0
I20260808 03:51:59.509903 5668 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:51:59.559675 4596 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55066:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:51:59.566553 4596 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:51:59.570636 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:59.570218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.0.0.1:35298)
0808 03:51:59.570362 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:59.570453 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 03:51:59.570837 4876 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [term 3 FOLLOWER]: Refusing update from remote peer d8fb352759794eaf894af22d78c9b648: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260808 03:51:59.571595 5665 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260808 03:51:59.571744 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:59.571300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.0.0.1:35310)
0808 03:51:59.571480 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:59.571632 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 03:51:59.571949 4577 consensus_peers.cc:487] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 -> Peer 4ae9871e1feb40edb677f7bb5a2cb131 (127.26.146.190:37325): Couldn't send request to peer 4ae9871e1feb40edb677f7bb5a2cb131. Status: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 03:51:59.574786 5670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8fb352759794eaf894af22d78c9b648. Latest consensus state: current_term: 3 leader_uuid: "d8fb352759794eaf894af22d78c9b648" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: false } } }
I20260808 03:51:59.575048 5670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:51:59.575861 5662 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "d8fb352759794eaf894af22d78c9b648" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: false } } }
I20260808 03:51:59.576187 5662 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648 [sys.catalog]: This master's current role is: LEADER
I20260808 03:51:59.576718 5670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "d8fb352759794eaf894af22d78c9b648" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4ae9871e1feb40edb677f7bb5a2cb131" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37325 } } peers { permanent_uuid: "d8fb352759794eaf894af22d78c9b648" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 41585 } attrs { promote: false } } peers { permanent_uuid: "9f6bdeb4f1c5459d87a2520c59d92e76" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 35251 } attrs { promote: false } } }
I20260808 03:51:59.576999 5670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f6bdeb4f1c5459d87a2520c59d92e76 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:51:59.591333 5677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:59.584096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37444 (local address 127.26.146.129:38971)
0808 03:51:59.585088 (+ 992us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:59.585095 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:59.585115 (+ 20us) server_negotiation.cc:407] Connection header received
0808 03:51:59.585186 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:59.585191 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:59.585309 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:59.586020 (+ 711us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:59.586225 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:59.587350 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:59.589341 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:59.590403 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:59.590582 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:59.590657 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:59.590920 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 03:51:59.591050 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":864,"thread_start_us":753,"threads_started":1}
I20260808 03:51:59.591333 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:59.583966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:38971 (local address 127.0.0.1:37444)
0808 03:51:59.584338 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:59.584366 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:59.584511 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:59.585477 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:59.585482 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:59.585509 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:59.585862 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:59.585877 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:59.587550 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:59.587556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:59.589189 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:59.589203 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:59.590585 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:59.590591 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:59.590769 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:59.590792 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 03:51:59.590872 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 03:51:59.591012 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234}
I20260808 03:51:59.592932 5018 tablet_service.cc:1467] Processing CreateTablet for tablet a781a6b2d91c411199a6b4eed0ae043d (DEFAULT_TABLE table=test-table [id=1619b580123b4b2eb623d5ffd1bba5f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:59.593112 5682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:59.584762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36396 (local address 127.26.146.131:37663)
0808 03:51:59.585285 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:59.585290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:59.585886 (+ 596us) server_negotiation.cc:407] Connection header received
0808 03:51:59.586045 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:59.586050 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:59.586143 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:59.586233 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:59.586888 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:59.587827 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:59.590532 (+ 2705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:59.591777 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:59.591955 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:59.592029 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:59.592628 (+ 599us) server_negotiation.cc:299] Negotiation successful
0808 03:51:59.592783 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":328,"threads_started":1}
I20260808 03:51:59.593402 5018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a781a6b2d91c411199a6b4eed0ae043d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:59.593868 5684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:59.584618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:37663 (local address 127.0.0.1:36396)
0808 03:51:59.585779 (+ 1161us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:59.585792 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:59.585918 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:59.586250 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:59.586254 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:59.586277 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:59.586672 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:59.586685 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:59.587954 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:59.587960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:59.590361 (+ 2401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:59.590377 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:59.592185 (+ 1808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:59.592191 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:59.592392 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:59.592429 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 03:51:59.593366 (+ 937us) client_negotiation.cc:240] Negotiation successful
0808 03:51:59.593513 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1056,"threads_started":1}
I20260808 03:51:59.597867 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:59.584623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:38737 (local address 127.0.0.1:41234)
0808 03:51:59.591565 (+ 6942us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:59.591581 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:59.591669 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:59.592059 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:59.592063 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:59.592083 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:59.592249 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:59.592260 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:59.593718 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:59.593723 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:59.595506 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:59.595521 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:59.597124 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:59.597130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:59.597324 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:59.597358 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 03:51:59.597438 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 03:51:59.597554 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6849,"mutex_wait_us":566,"thread_start_us":326,"threads_started":1}
I20260808 03:51:59.597927 5683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:59.585027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41234 (local address 127.26.146.130:38737)
0808 03:51:59.585636 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:59.585641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:59.591764 (+ 6123us) server_negotiation.cc:407] Connection header received
0808 03:51:59.591860 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:59.591864 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:59.591928 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:59.592403 (+ 475us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:59.592578 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:59.593544 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:59.595724 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:59.596939 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:59.597119 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:59.597188 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:59.597490 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 03:51:59.597624 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":376,"threads_started":1}
I20260808 03:51:59.597900 5454 tablet_service.cc:1467] Processing CreateTablet for tablet a781a6b2d91c411199a6b4eed0ae043d (DEFAULT_TABLE table=test-table [id=1619b580123b4b2eb623d5ffd1bba5f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:59.598356 5454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a781a6b2d91c411199a6b4eed0ae043d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:59.598619 5241 tablet_service.cc:1467] Processing CreateTablet for tablet a781a6b2d91c411199a6b4eed0ae043d (DEFAULT_TABLE table=test-table [id=1619b580123b4b2eb623d5ffd1bba5f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:51:59.598949 5241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a781a6b2d91c411199a6b4eed0ae043d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:51:59.604002 5687 tablet_bootstrap.cc:492] T a781a6b2d91c411199a6b4eed0ae043d P d330735cea094acfaa5238dab0764ce1: Bootstrap starting.
I20260808 03:51:59.605378 5688 tablet_bootstrap.cc:492] T a781a6b2d91c411199a6b4eed0ae043d P d02e63de991343c891c7a1054e10471d: Bootstrap starting.
I20260808 03:51:59.606125 5686 tablet_bootstrap.cc:492] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6: Bootstrap starting.
I20260808 03:51:59.606986 5687 tablet_bootstrap.cc:654] T a781a6b2d91c411199a6b4eed0ae043d P d330735cea094acfaa5238dab0764ce1: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:59.607120 5688 tablet_bootstrap.cc:654] T a781a6b2d91c411199a6b4eed0ae043d P d02e63de991343c891c7a1054e10471d: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:59.607668 5686 tablet_bootstrap.cc:654] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6: Neither blocks nor log segments found. Creating new log.
I20260808 03:51:59.609799 5686 tablet_bootstrap.cc:492] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6: No bootstrap required, opened a new log
I20260808 03:51:59.610010 5686 ts_tablet_manager.cc:1397] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 03:51:59.610692 5686 raft_consensus.cc:348] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 38971 } } peers { permanent_uuid: "d330735cea094acfaa5238dab0764ce1" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38737 } } peers { permanent_uuid: "d02e63de991343c891c7a1054e10471d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37663 } }
I20260808 03:51:59.610893 5686 raft_consensus.cc:374] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:59.611011 5686 raft_consensus.cc:729] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 502e92ad9ce04a7ba76a5e857a7e51d6, State: Initialized, Role: FOLLOWER
I20260808 03:51:59.611276 5686 consensus_queue.cc:260] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 38971 } } peers { permanent_uuid: "d330735cea094acfaa5238dab0764ce1" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38737 } } peers { permanent_uuid: "d02e63de991343c891c7a1054e10471d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37663 } }
I20260808 03:51:59.612071 5686 ts_tablet_manager.cc:1428] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 03:51:59.615366 5688 tablet_bootstrap.cc:492] T a781a6b2d91c411199a6b4eed0ae043d P d02e63de991343c891c7a1054e10471d: No bootstrap required, opened a new log
I20260808 03:51:59.615571 5688 ts_tablet_manager.cc:1397] T a781a6b2d91c411199a6b4eed0ae043d P d02e63de991343c891c7a1054e10471d: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.009s
I20260808 03:51:59.616453 5688 raft_consensus.cc:348] T a781a6b2d91c411199a6b4eed0ae043d P d02e63de991343c891c7a1054e10471d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 38971 } } peers { permanent_uuid: "d330735cea094acfaa5238dab0764ce1" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38737 } } peers { permanent_uuid: "d02e63de991343c891c7a1054e10471d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37663 } }
I20260808 03:51:59.616688 5688 raft_consensus.cc:374] T a781a6b2d91c411199a6b4eed0ae043d P d02e63de991343c891c7a1054e10471d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:59.616729 5687 tablet_bootstrap.cc:492] T a781a6b2d91c411199a6b4eed0ae043d P d330735cea094acfaa5238dab0764ce1: No bootstrap required, opened a new log
I20260808 03:51:59.616863 5687 ts_tablet_manager.cc:1397] T a781a6b2d91c411199a6b4eed0ae043d P d330735cea094acfaa5238dab0764ce1: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.003s
I20260808 03:51:59.616842 5688 raft_consensus.cc:729] T a781a6b2d91c411199a6b4eed0ae043d P d02e63de991343c891c7a1054e10471d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e63de991343c891c7a1054e10471d, State: Initialized, Role: FOLLOWER
I20260808 03:51:59.617110 5688 consensus_queue.cc:260] T a781a6b2d91c411199a6b4eed0ae043d P d02e63de991343c891c7a1054e10471d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 38971 } } peers { permanent_uuid: "d330735cea094acfaa5238dab0764ce1" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38737 } } peers { permanent_uuid: "d02e63de991343c891c7a1054e10471d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37663 } }
I20260808 03:51:59.617619 5687 raft_consensus.cc:348] T a781a6b2d91c411199a6b4eed0ae043d P d330735cea094acfaa5238dab0764ce1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 38971 } } peers { permanent_uuid: "d330735cea094acfaa5238dab0764ce1" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38737 } } peers { permanent_uuid: "d02e63de991343c891c7a1054e10471d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37663 } }
I20260808 03:51:59.617933 5687 raft_consensus.cc:374] T a781a6b2d91c411199a6b4eed0ae043d P d330735cea094acfaa5238dab0764ce1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:51:59.618350 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:59.612637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.129:54145)
0808 03:51:59.617894 (+ 5257us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:59.618060 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5120,"thread_start_us":4995,"threads_started":1}
I20260808 03:51:59.619477 5688 ts_tablet_manager.cc:1428] T a781a6b2d91c411199a6b4eed0ae043d P d02e63de991343c891c7a1054e10471d: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260808 03:51:59.620249 5687 raft_consensus.cc:729] T a781a6b2d91c411199a6b4eed0ae043d P d330735cea094acfaa5238dab0764ce1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d330735cea094acfaa5238dab0764ce1, State: Initialized, Role: FOLLOWER
I20260808 03:51:59.620627 5687 consensus_queue.cc:260] T a781a6b2d91c411199a6b4eed0ae043d P d330735cea094acfaa5238dab0764ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 38971 } } peers { permanent_uuid: "d330735cea094acfaa5238dab0764ce1" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38737 } } peers { permanent_uuid: "d02e63de991343c891c7a1054e10471d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37663 } }
I20260808 03:51:59.622745 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:59.618971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.129:59397)
0808 03:51:59.622494 (+ 3523us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:59.622583 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3426,"threads_started":1}
I20260808 03:51:59.624626 5120 heartbeater.cc:503] Master 127.26.146.189:41585 was elected leader, sending a full tablet report...
I20260808 03:51:59.624676 5698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:59.618772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.131:40945)
0808 03:51:59.624294 (+ 5522us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:59.624430 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5389,"thread_start_us":5278,"threads_started":1}
I20260808 03:51:59.625275 5687 ts_tablet_manager.cc:1428] T a781a6b2d91c411199a6b4eed0ae043d P d330735cea094acfaa5238dab0764ce1: Time spent starting tablet: real 0.008s user 0.002s sys 0.004s
I20260808 03:51:59.625937 5544 heartbeater.cc:503] Master 127.26.146.189:41585 was elected leader, sending a full tablet report...
I20260808 03:51:59.628391 5698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:59.626566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.131:60365)
0808 03:51:59.628217 (+ 1651us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:59.628293 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1569}
I20260808 03:51:59.629253 5701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:59.625837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.130:46515)
0808 03:51:59.628868 (+ 3031us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:59.629008 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2932,"thread_start_us":2806,"threads_started":1}
I20260808 03:51:59.630159 5701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:59.629833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.130:50375)
0808 03:51:59.629970 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:59.630055 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 03:51:59.630184 5335 heartbeater.cc:503] Master 127.26.146.189:41585 was elected leader, sending a full tablet report...
W20260808 03:51:59.710537 5545 tablet.cc:2369] T a781a6b2d91c411199a6b4eed0ae043d P d02e63de991343c891c7a1054e10471d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:51:59.758662 5122 tablet.cc:2369] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:51:59.856206 5336 tablet.cc:2369] T a781a6b2d91c411199a6b4eed0ae043d P d330735cea094acfaa5238dab0764ce1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:51:59.981468 5694 raft_consensus.cc:482] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:51:59.981741 5694 raft_consensus.cc:504] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 38971 } } peers { permanent_uuid: "d330735cea094acfaa5238dab0764ce1" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38737 } } peers { permanent_uuid: "d02e63de991343c891c7a1054e10471d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37663 } }
I20260808 03:51:59.983417 5694 leader_election.cc:288] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d330735cea094acfaa5238dab0764ce1 (127.26.146.130:38737), d02e63de991343c891c7a1054e10471d (127.26.146.131:37663)
I20260808 03:51:59.991376 5683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:59.983842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:34805 (local address 127.26.146.130:38737)
0808 03:51:59.984147 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:59.984153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:59.984177 (+ 24us) server_negotiation.cc:407] Connection header received
0808 03:51:59.984267 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:59.984271 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:59.984339 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:59.984435 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:59.985052 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:59.986186 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:59.989068 (+ 2882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:59.990309 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:59.990459 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:59.990539 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:59.990982 (+ 443us) server_negotiation.cc:299] Negotiation successful
0808 03:51:59.991111 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260808 03:51:59.991385 5682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:59.984005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:58645 (local address 127.26.146.131:37663)
0808 03:51:59.984196 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 03:51:59.984202 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:51:59.984345 (+ 143us) server_negotiation.cc:407] Connection header received
0808 03:51:59.984494 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:51:59.984498 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:51:59.984569 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:51:59.984641 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:51:59.985153 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:59.986196 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:59.989058 (+ 2862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:51:59.990305 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:51:59.990462 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:59.990536 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 03:51:59.990982 (+ 446us) server_negotiation.cc:299] Negotiation successful
0808 03:51:59.991110 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 03:51:59.991446 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:59.983752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:37663 (local address 127.26.146.129:58645)
0808 03:51:59.984252 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:59.984268 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:59.984370 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:59.984692 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:59.984697 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:59.984719 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:59.985024 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:59.985035 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:59.986330 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:59.986335 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:59.988893 (+ 2558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:59.988911 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:59.990518 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:59.990526 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:59.990807 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:59.990847 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 03:51:59.990946 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 03:51:59.991099 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":343}
I20260808 03:51:59.991470 5699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:51:59.983735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:38737 (local address 127.26.146.129:34805)
0808 03:51:59.983952 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 03:51:59.983972 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:51:59.984134 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:51:59.984476 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:51:59.984480 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:51:59.984499 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:51:59.984846 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:59.984858 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:59.986330 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:59.986335 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:59.988899 (+ 2564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:51:59.988911 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:51:59.990563 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:51:59.990569 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:51:59.990801 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:51:59.990849 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 03:51:59.990945 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 03:51:59.991099 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":218}
I20260808 03:51:59.992205 5270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a781a6b2d91c411199a6b4eed0ae043d" candidate_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d330735cea094acfaa5238dab0764ce1" is_pre_election: true
I20260808 03:51:59.992488 5270 raft_consensus.cc:2454] T a781a6b2d91c411199a6b4eed0ae043d P d330735cea094acfaa5238dab0764ce1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 502e92ad9ce04a7ba76a5e857a7e51d6 in term 0.
I20260808 03:51:59.992547 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a781a6b2d91c411199a6b4eed0ae043d" candidate_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e63de991343c891c7a1054e10471d" is_pre_election: true
I20260808 03:51:59.992990 5479 raft_consensus.cc:2454] T a781a6b2d91c411199a6b4eed0ae043d P d02e63de991343c891c7a1054e10471d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 502e92ad9ce04a7ba76a5e857a7e51d6 in term 0.
I20260808 03:51:59.993088 4956 leader_election.cc:302] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 502e92ad9ce04a7ba76a5e857a7e51d6, d330735cea094acfaa5238dab0764ce1; no voters:
I20260808 03:51:59.993474 5694 raft_consensus.cc:2781] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:51:59.993584 5694 raft_consensus.cc:482] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:51:59.993680 5694 raft_consensus.cc:3037] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:59.995785 5694 raft_consensus.cc:504] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 38971 } } peers { permanent_uuid: "d330735cea094acfaa5238dab0764ce1" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38737 } } peers { permanent_uuid: "d02e63de991343c891c7a1054e10471d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37663 } }
I20260808 03:51:59.996431 5694 leader_election.cc:288] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6 [CANDIDATE]: Term 1 election: Requested vote from peers d330735cea094acfaa5238dab0764ce1 (127.26.146.130:38737), d02e63de991343c891c7a1054e10471d (127.26.146.131:37663)
I20260808 03:51:59.996953 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a781a6b2d91c411199a6b4eed0ae043d" candidate_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e63de991343c891c7a1054e10471d"
I20260808 03:51:59.996971 5270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a781a6b2d91c411199a6b4eed0ae043d" candidate_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d330735cea094acfaa5238dab0764ce1"
I20260808 03:51:59.997174 5479 raft_consensus.cc:3037] T a781a6b2d91c411199a6b4eed0ae043d P d02e63de991343c891c7a1054e10471d [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:51:59.997174 5270 raft_consensus.cc:3037] T a781a6b2d91c411199a6b4eed0ae043d P d330735cea094acfaa5238dab0764ce1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:52:00.000095 5270 raft_consensus.cc:2454] T a781a6b2d91c411199a6b4eed0ae043d P d330735cea094acfaa5238dab0764ce1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 502e92ad9ce04a7ba76a5e857a7e51d6 in term 1.
I20260808 03:52:00.000108 5479 raft_consensus.cc:2454] T a781a6b2d91c411199a6b4eed0ae043d P d02e63de991343c891c7a1054e10471d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 502e92ad9ce04a7ba76a5e857a7e51d6 in term 1.
I20260808 03:52:00.000620 4958 leader_election.cc:302] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 502e92ad9ce04a7ba76a5e857a7e51d6, d02e63de991343c891c7a1054e10471d; no voters:
I20260808 03:52:00.000941 5694 raft_consensus.cc:2781] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:52:00.001377 5694 raft_consensus.cc:686] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6 [term 1 LEADER]: Becoming Leader. State: Replica: 502e92ad9ce04a7ba76a5e857a7e51d6, State: Running, Role: LEADER
I20260808 03:52:00.001801 5694 consensus_queue.cc:237] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 38971 } } peers { permanent_uuid: "d330735cea094acfaa5238dab0764ce1" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38737 } } peers { permanent_uuid: "d02e63de991343c891c7a1054e10471d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37663 } }
I20260808 03:52:00.002424 5699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:00.001996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.129:44773)
0808 03:52:00.002180 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:00.002300 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 03:52:00.005376 4596 catalog_manager.cc:5495] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 502e92ad9ce04a7ba76a5e857a7e51d6 (127.26.146.129). New cstate: current_term: 1 leader_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "502e92ad9ce04a7ba76a5e857a7e51d6" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 38971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d330735cea094acfaa5238dab0764ce1" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e63de991343c891c7a1054e10471d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37663 } health_report { overall_health: UNKNOWN } } }
I20260808 03:52:00.047173 5677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:00.041671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37456 (local address 127.26.146.129:38971)
0808 03:52:00.041844 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:00.041848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:00.041866 (+ 18us) server_negotiation.cc:407] Connection header received
0808 03:52:00.042105 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:00.042109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:00.042199 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:00.042327 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:52:00.042651 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:00.043697 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:00.045259 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:00.045686 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:00.045775 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:00.046136 (+ 361us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:52:00.046681 (+ 545us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:52:00.046792 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:00.046888 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 03:52:00.046955 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 03:52:00.051433 5270 raft_consensus.cc:1261] T a781a6b2d91c411199a6b4eed0ae043d P d330735cea094acfaa5238dab0764ce1 [term 1 FOLLOWER]: Refusing update from remote peer 502e92ad9ce04a7ba76a5e857a7e51d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:52:00.051844 5479 raft_consensus.cc:1261] T a781a6b2d91c411199a6b4eed0ae043d P d02e63de991343c891c7a1054e10471d [term 1 FOLLOWER]: Refusing update from remote peer 502e92ad9ce04a7ba76a5e857a7e51d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:52:00.052212 5702 consensus_queue.cc:1035] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d330735cea094acfaa5238dab0764ce1" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:52:00.052601 5694 consensus_queue.cc:1035] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e63de991343c891c7a1054e10471d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:52:00.056823 5701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:00.055934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.130:33465)
0808 03:52:00.056527 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:00.056658 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478}
I20260808 03:52:00.056828 5698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:00.056292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.131:37885)
0808 03:52:00.056504 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:00.056633 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 03:52:00.064139 5698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:00.063565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.131:46181)
0808 03:52:00.063842 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:00.063937 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260808 03:52:00.073284 5699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:00.072619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.129:34659)
0808 03:52:00.072967 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:00.073073 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219}
I20260808 03:52:00.077873 5699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:00.077459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.129:58953)
0808 03:52:00.077609 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:00.077722 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 03:52:00.080309 5701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:00.079690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.26.146.130:45533)
0808 03:52:00.080075 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:00.080162 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270}
I20260808 03:52:00.087677 4596 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:52:00.087952 4596 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:52:00.093636 4596 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d8fb352759794eaf894af22d78c9b648: Sending DeleteTablet for 3 replicas of tablet a781a6b2d91c411199a6b4eed0ae043d
I20260808 03:52:00.094822 5241 tablet_service.cc:1514] Processing DeleteTablet for tablet a781a6b2d91c411199a6b4eed0ae043d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:52:00 UTC) from {username='oryx', principal='oryx/127.26.146.189@KRBTEST.COM'} at 127.0.0.1:41234
I20260808 03:52:00.094826 5018 tablet_service.cc:1514] Processing DeleteTablet for tablet a781a6b2d91c411199a6b4eed0ae043d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:52:00 UTC) from {username='oryx', principal='oryx/127.26.146.189@KRBTEST.COM'} at 127.0.0.1:37444
I20260808 03:52:00.095530 5727 tablet_replica.cc:330] T a781a6b2d91c411199a6b4eed0ae043d P d330735cea094acfaa5238dab0764ce1: stopping tablet replica
I20260808 03:52:00.095775 5727 raft_consensus.cc:2229] T a781a6b2d91c411199a6b4eed0ae043d P d330735cea094acfaa5238dab0764ce1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:52:00.096225 5727 raft_consensus.cc:2258] T a781a6b2d91c411199a6b4eed0ae043d P d330735cea094acfaa5238dab0764ce1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:52:00.096185 5454 tablet_service.cc:1514] Processing DeleteTablet for tablet a781a6b2d91c411199a6b4eed0ae043d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:52:00 UTC) from {username='oryx', principal='oryx/127.26.146.189@KRBTEST.COM'} at 127.0.0.1:36396
I20260808 03:52:00.097652 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 4831
I20260808 03:52:00.108471 5728 tablet_replica.cc:330] T a781a6b2d91c411199a6b4eed0ae043d P d02e63de991343c891c7a1054e10471d: stopping tablet replica
I20260808 03:52:00.108786 5728 raft_consensus.cc:2229] T a781a6b2d91c411199a6b4eed0ae043d P d02e63de991343c891c7a1054e10471d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:52:00.109208 5728 raft_consensus.cc:2258] T a781a6b2d91c411199a6b4eed0ae043d P d02e63de991343c891c7a1054e10471d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:52:00.111002 5728 ts_tablet_manager.cc:1905] T a781a6b2d91c411199a6b4eed0ae043d P d02e63de991343c891c7a1054e10471d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:52:00.111651 5727 ts_tablet_manager.cc:1905] T a781a6b2d91c411199a6b4eed0ae043d P d330735cea094acfaa5238dab0764ce1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:52:00.118113 5728 ts_tablet_manager.cc:1918] T a781a6b2d91c411199a6b4eed0ae043d P d02e63de991343c891c7a1054e10471d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:52:00.118124 5727 ts_tablet_manager.cc:1918] T a781a6b2d91c411199a6b4eed0ae043d P d330735cea094acfaa5238dab0764ce1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:52:00.118819 5725 tablet_replica.cc:330] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6: stopping tablet replica
I20260808 03:52:00.119132 5725 raft_consensus.cc:2229] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:52:00.119796 5725 raft_consensus.cc:2258] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:52:00.120468 5728 log.cc:1196] T a781a6b2d91c411199a6b4eed0ae043d P d02e63de991343c891c7a1054e10471d: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/a781a6b2d91c411199a6b4eed0ae043d
I20260808 03:52:00.120959 5728 ts_tablet_manager.cc:1939] T a781a6b2d91c411199a6b4eed0ae043d P d02e63de991343c891c7a1054e10471d: Deleting consensus metadata
I20260808 03:52:00.122249 4580 catalog_manager.cc:4841] TS d02e63de991343c891c7a1054e10471d (127.26.146.131:37663): tablet a781a6b2d91c411199a6b4eed0ae043d (table test-table [id=1619b580123b4b2eb623d5ffd1bba5f7]) successfully deleted
I20260808 03:52:00.122740 5725 ts_tablet_manager.cc:1905] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:52:00.128793 5727 log.cc:1196] T a781a6b2d91c411199a6b4eed0ae043d P d330735cea094acfaa5238dab0764ce1: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/a781a6b2d91c411199a6b4eed0ae043d
I20260808 03:52:00.129312 5727 ts_tablet_manager.cc:1939] T a781a6b2d91c411199a6b4eed0ae043d P d330735cea094acfaa5238dab0764ce1: Deleting consensus metadata
I20260808 03:52:00.129624 5725 ts_tablet_manager.cc:1918] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:52:00.129772 5725 log.cc:1196] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786161058098901-27210-0/minicluster-data/ts-0/wal/wals/a781a6b2d91c411199a6b4eed0ae043d
I20260808 03:52:00.130214 5725 ts_tablet_manager.cc:1939] T a781a6b2d91c411199a6b4eed0ae043d P 502e92ad9ce04a7ba76a5e857a7e51d6: Deleting consensus metadata
I20260808 03:52:00.130586 4578 catalog_manager.cc:4841] TS d330735cea094acfaa5238dab0764ce1 (127.26.146.130:38737): tablet a781a6b2d91c411199a6b4eed0ae043d (table test-table [id=1619b580123b4b2eb623d5ffd1bba5f7]) successfully deleted
I20260808 03:52:00.131367 4579 catalog_manager.cc:4841] TS 502e92ad9ce04a7ba76a5e857a7e51d6 (127.26.146.129:38971): tablet a781a6b2d91c411199a6b4eed0ae043d (table test-table [id=1619b580123b4b2eb623d5ffd1bba5f7]) successfully deleted
I20260808 03:52:00.136122 5677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:00.099373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37472 (local address 127.26.146.129:38971)
0808 03:52:00.099558 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:00.099565 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:00.108789 (+ 9224us) server_negotiation.cc:407] Connection header received
0808 03:52:00.109783 (+ 994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:00.109789 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:00.109904 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:00.109982 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:00.119948 (+ 9966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:00.121158 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:00.123223 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:00.123973 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:00.124115 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:00.127699 (+ 3584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:00.127720 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:00.127745 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:00.127797 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:00.133029 (+ 5232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:00.133834 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:00.133844 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:00.133847 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:00.134040 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:00.134460 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:00.134469 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:00.134473 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:00.134953 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:00.135275 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:00.135481 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 03:52:00.135706 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 03:52:00.136801 5112 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:00.188517 5731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:00.187768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.0.0.1:35318)
0808 03:52:00.188243 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:00.188336 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":219,"threads_started":1}
I20260808 03:52:00.190018 5731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:00.189658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.0.0.1:35320)
0808 03:52:00.189818 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:00.189893 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 03:52:00.239042 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 4831
I20260808 03:52:00.264443 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 5133
I20260808 03:52:00.276655 5683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:00.265216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41242 (local address 127.26.146.130:38737)
0808 03:52:00.265386 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:00.265391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:00.265426 (+ 35us) server_negotiation.cc:407] Connection header received
0808 03:52:00.265571 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:00.265575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:00.265664 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:00.265743 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:00.266988 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:00.267724 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:00.268885 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:00.269272 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:00.269364 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:00.271724 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:00.271740 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:00.271757 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:00.271796 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:00.274396 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:00.274977 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:00.274984 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:00.274986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:00.275110 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:00.275397 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:00.275399 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:00.275401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:00.275763 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:00.276034 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:00.276212 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 03:52:00.276387 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 03:52:00.277311 5327 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:00.381947 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 5133
I20260808 03:52:00.406497 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 5349
I20260808 03:52:00.420051 5682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:00.407235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36398 (local address 127.26.146.131:37663)
0808 03:52:00.407427 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:00.407434 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:00.407455 (+ 21us) server_negotiation.cc:407] Connection header received
0808 03:52:00.407728 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:00.407733 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:00.407820 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:00.407912 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:00.409335 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:00.410141 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:00.411350 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:00.411764 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:00.411938 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:00.414372 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:00.414388 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:00.414405 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:00.414438 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:00.417283 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:00.417962 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:00.417970 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:00.417972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:00.418100 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:00.418468 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:00.418472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:00.418474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:00.418837 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:00.419130 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:00.419451 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 03:52:00.419673 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 03:52:00.420740 5534 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:00.537371 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 5349
I20260808 03:52:00.564090 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 4404
I20260808 03:52:00.565292 4636 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:00.686620 5731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:00.686162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37325 (local address 127.0.0.1:35326)
0808 03:52:00.686337 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:00.686420 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 03:52:00.686812 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 4404
I20260808 03:52:00.712896 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 4548
I20260808 03:52:00.713928 4901 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:00.831966 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 4548
2026-08-08T03:52:00Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11858 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:52:00 dist-test-slave-jlzn krb5kdc[5740](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:52:00 dist-test-slave-jlzn krb5kdc[5740](info): set up 2 sockets
Aug 08 03:52:00 dist-test-slave-jlzn krb5kdc[5740](info): commencing operation
krb5kdc: starting...
W20260808 03:52:02.039340 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:52:02 dist-test-slave-jlzn krb5kdc[5740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:52:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:52:02Z Disabled control of system clock
WARNING: no policy specified for oryx/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.190.keytab.
Entry for principal oryx/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.190.keytab.
I20260808 03:52:02.286945 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:42201
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.190.keytab
--principal=oryx/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:46519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:42201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:52:02.608860 5757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:02.609135 5757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:02.609231 5757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:02.616235 5757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:52:02.616334 5757 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 03:52:02.616380 5757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:02.616441 5757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:02.616490 5757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:52:02.616556 5757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:52:02.626072 5757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:46519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:42201
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.190.keytab
--principal=oryx/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:42201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:02.627220 5757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:02.628515 5757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:02.639737 5762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:02.639770 5770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:02.639819 5767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:02.640568 5757 server_base.cc:1034] running on GCE node
I20260808 03:52:02.641232 5757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:02.642480 5757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:02.643703 5757 hybrid_clock.cc:648] HybridClock initialized: now 1786161122643640 us; error 33 us; skew 500 ppm
I20260808 03:52:02.644032 5757 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:52:02 dist-test-slave-jlzn krb5kdc[5740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161122, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:52:02.646901 5757 init.cc:377] Logged in from keytab as oryx/127.26.146.190@KRBTEST.COM (short username oryx)
I20260808 03:52:02.648365 5757 webserver.cc:460] Webserver started at http://127.26.146.190:41981/ using document root <none> and password file <none>
I20260808 03:52:02.649031 5757 fs_manager.cc:359] Metadata directory not provided
I20260808 03:52:02.649138 5757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:52:02.649452 5757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:52:02.652413 5757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "9c7ea7ea0cf54c879112453ed0d0ed47"
format_stamp: "Formatted at 2026-08-08 03:52:02 on dist-test-slave-jlzn"
server_key: "77148abc3d49f5d0b1887c8d52518ba6"
server_key_iv: "0d5b20b6a5b722523ddf5559e6658729"
server_key_version: "encryptionkey@0"
I20260808 03:52:02.653205 5757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "9c7ea7ea0cf54c879112453ed0d0ed47"
format_stamp: "Formatted at 2026-08-08 03:52:02 on dist-test-slave-jlzn"
server_key: "77148abc3d49f5d0b1887c8d52518ba6"
server_key_iv: "0d5b20b6a5b722523ddf5559e6658729"
server_key_version: "encryptionkey@0"
I20260808 03:52:02.658288 5757 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:52:02.661681 5778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:02.662922 5757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 03:52:02.663074 5757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "9c7ea7ea0cf54c879112453ed0d0ed47"
format_stamp: "Formatted at 2026-08-08 03:52:02 on dist-test-slave-jlzn"
server_key: "77148abc3d49f5d0b1887c8d52518ba6"
server_key_iv: "0d5b20b6a5b722523ddf5559e6658729"
server_key_version: "encryptionkey@0"
I20260808 03:52:02.663273 5757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:52:02.678153 5757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:52:02.682063 5757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:52:02.682308 5757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:52:02.702644 5757 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:42201
I20260808 03:52:02.702664 5870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:42201 every 8 connection(s)
I20260808 03:52:02.704041 5757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:52:02.707721 5871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:52:02.709002 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 5757
I20260808 03:52:02.709231 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:52:02.710059 27210 external_mini_cluster.cc:1428] Setting key 5d3ea0961763dffa9ba256a7787ba18c
I20260808 03:52:02.721235 5871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47: Bootstrap starting.
I20260808 03:52:02.724860 5871 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47: Neither blocks nor log segments found. Creating new log.
I20260808 03:52:02.725842 5871 log.cc:824] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47: Log is configured to *not* fsync() on all Append() calls
Aug 08 03:52:02 dist-test-slave-jlzn krb5kdc[5740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM
I20260808 03:52:02.728818 5871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47: No bootstrap required, opened a new log
I20260808 03:52:02.735014 5874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:02.712419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47648 (local address 127.26.146.190:42201)
0808 03:52:02.713507 (+ 1088us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:02.713522 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:02.713966 (+ 444us) server_negotiation.cc:407] Connection header received
0808 03:52:02.716920 (+ 2954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:02.716950 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:02.717443 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:02.717900 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:02.719166 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:02.721140 (+ 1974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:02.722441 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:02.723511 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:02.723698 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:02.727796 (+ 4098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:02.727829 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:02.727883 (+ 54us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:02.727942 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:02.730953 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:02.731635 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:02.731645 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:02.731651 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:02.731864 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:02.732252 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:02.732257 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:02.732260 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:02.732811 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:02.733131 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:02.733582 (+ 451us) server_negotiation.cc:299] Negotiation successful
0808 03:52:02.734002 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":821,"thread_start_us":536,"threads_started":1}
I20260808 03:52:02.736761 5871 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c7ea7ea0cf54c879112453ed0d0ed47" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 42201 } }
I20260808 03:52:02.737120 5871 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:52:02.737221 5871 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c7ea7ea0cf54c879112453ed0d0ed47, State: Initialized, Role: FOLLOWER
I20260808 03:52:02.737833 5871 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c7ea7ea0cf54c879112453ed0d0ed47" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 42201 } }
I20260808 03:52:02.738077 5871 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:52:02.738211 5871 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:52:02.738360 5871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:52:02.740525 5871 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c7ea7ea0cf54c879112453ed0d0ed47" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 42201 } }
I20260808 03:52:02.740998 5871 leader_election.cc:302] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c7ea7ea0cf54c879112453ed0d0ed47; no voters:
I20260808 03:52:02.741531 5871 leader_election.cc:288] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:52:02.741648 5877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:52:02.742947 5877 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47 [term 1 LEADER]: Becoming Leader. State: Replica: 9c7ea7ea0cf54c879112453ed0d0ed47, State: Running, Role: LEADER
I20260808 03:52:02.743531 5877 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c7ea7ea0cf54c879112453ed0d0ed47" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 42201 } }
I20260808 03:52:02.743741 5871 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:52:02.746520 5880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c7ea7ea0cf54c879112453ed0d0ed47. Latest consensus state: current_term: 1 leader_uuid: "9c7ea7ea0cf54c879112453ed0d0ed47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c7ea7ea0cf54c879112453ed0d0ed47" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 42201 } } }
I20260808 03:52:02.746548 5879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c7ea7ea0cf54c879112453ed0d0ed47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c7ea7ea0cf54c879112453ed0d0ed47" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 42201 } } }
I20260808 03:52:02.747018 5879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47 [sys.catalog]: This master's current role is: LEADER
I20260808 03:52:02.747018 5880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47 [sys.catalog]: This master's current role is: LEADER
I20260808 03:52:02.747931 5890 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:52:02.753419 5890 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:52:02.762753 5890 catalog_manager.cc:1347] Generated new cluster ID: 5865cbc66c90410b9d521c4f0adf5bc6
I20260808 03:52:02.762876 5890 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:52:02.798156 5890 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:52:02.799552 5890 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:52:02.809320 5890 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47: Generated new TSK 0
I20260808 03:52:02.810163 5890 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.26.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.129.keytab.
Entry for principal oryx/127.26.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.129.keytab.
I20260808 03:52:02.879340 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:42201
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.129.keytab
--principal=oryx/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:46519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:52:03.261650 5906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:03.261929 5906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:03.262068 5906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:03.269412 5906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:03.269523 5906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:03.269647 5906 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 03:52:03.269757 5906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:52:03.279326 5906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:46519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.129.keytab
--principal=oryx/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.146.190:42201
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:03.280647 5906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:03.281982 5906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:03.294644 5914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:03.294685 5912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:03.294713 5916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:03.295414 5906 server_base.cc:1034] running on GCE node
I20260808 03:52:03.296075 5906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:03.296957 5906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:03.298161 5906 hybrid_clock.cc:648] HybridClock initialized: now 1786161123298086 us; error 48 us; skew 500 ppm
I20260808 03:52:03.298445 5906 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:52:03 dist-test-slave-jlzn krb5kdc[5740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161123, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:52:03.301512 5906 init.cc:377] Logged in from keytab as oryx/127.26.146.129@KRBTEST.COM (short username oryx)
I20260808 03:52:03.302933 5906 webserver.cc:460] Webserver started at http://127.26.146.129:33857/ using document root <none> and password file <none>
I20260808 03:52:03.303596 5906 fs_manager.cc:359] Metadata directory not provided
I20260808 03:52:03.303709 5906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:52:03.304061 5906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:52:03.306937 5906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "a566c5a4d4494120ac546c23153d5e04"
format_stamp: "Formatted at 2026-08-08 03:52:03 on dist-test-slave-jlzn"
server_key: "5ee8e7c87852fa7559fab4a0157f400b"
server_key_iv: "52ee38f519a9202735d36e658f70b1b3"
server_key_version: "encryptionkey@0"
I20260808 03:52:03.307830 5906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "a566c5a4d4494120ac546c23153d5e04"
format_stamp: "Formatted at 2026-08-08 03:52:03 on dist-test-slave-jlzn"
server_key: "5ee8e7c87852fa7559fab4a0157f400b"
server_key_iv: "52ee38f519a9202735d36e658f70b1b3"
server_key_version: "encryptionkey@0"
I20260808 03:52:03.313131 5906 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 03:52:03.316563 5927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:03.317832 5906 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:52:03.317993 5906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "a566c5a4d4494120ac546c23153d5e04"
format_stamp: "Formatted at 2026-08-08 03:52:03 on dist-test-slave-jlzn"
server_key: "5ee8e7c87852fa7559fab4a0157f400b"
server_key_iv: "52ee38f519a9202735d36e658f70b1b3"
server_key_version: "encryptionkey@0"
I20260808 03:52:03.318210 5906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:52:03.331844 5906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:52:03.335439 5906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:52:03.335696 5906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:52:03.338362 5906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:52:03.338469 5906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:03.338624 5906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:52:03.338738 5906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 08 03:52:03 dist-test-slave-jlzn krb5kdc[5740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161123, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM
I20260808 03:52:03.360316 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:03.338728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:55493 (local address 127.26.146.190:42201)
0808 03:52:03.339176 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:03.339182 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:03.340348 (+ 1166us) server_negotiation.cc:407] Connection header received
0808 03:52:03.342369 (+ 2021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:03.342374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:03.342468 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:03.342593 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:03.344602 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:03.345767 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:03.348411 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:03.348862 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:03.348980 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:03.352989 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:03.352996 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:03.353000 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:03.353043 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:03.356405 (+ 3362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:03.357480 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:03.357486 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:03.357489 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:03.357624 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:03.358221 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:03.358227 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:03.358230 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:03.358564 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:03.358722 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:03.359695 (+ 973us) server_negotiation.cc:299] Negotiation successful
0808 03:52:03.359943 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":232,"threads_started":1}
I20260808 03:52:03.361310 5943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:03.339386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:42201 (local address 127.26.146.129:55493)
0808 03:52:03.340189 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:03.340228 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:03.342038 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:03.342792 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:03.342800 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:03.343426 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:03.344335 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:03.344359 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:03.345932 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:03.345938 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:03.348265 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:03.348284 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:03.349098 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:03.349105 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:03.349322 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:03.349957 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:03.349978 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:03.352663 (+ 2685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:52:03.356569 (+ 3906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:03.356582 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:03.356603 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:03.357331 (+ 728us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:03.357745 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:03.357753 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:03.357757 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:03.358085 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:03.358792 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:52:03.358798 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:52:03.359246 (+ 448us) client_negotiation.cc:769] Sending connection context
0808 03:52:03.359557 (+ 311us) client_negotiation.cc:240] Negotiation successful
0808 03:52:03.360140 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":347,"threads_started":1}
I20260808 03:52:03.380954 6041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:03.367481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:42201 (local address 127.26.146.129:35109)
0808 03:52:03.368145 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:03.368169 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:03.368325 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:03.368678 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:03.368682 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:03.369092 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:03.369370 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:03.369379 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:03.370772 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:03.370776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:03.371790 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:03.371826 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:03.372666 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:03.372673 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:03.372917 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:03.373669 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:03.373701 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:03.375025 (+ 1324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:52:03.378283 (+ 3258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:03.378289 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:03.378293 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:03.378888 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:03.379265 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:03.379270 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:03.379274 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:03.379397 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:03.379937 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:52:03.379942 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:52:03.380144 (+ 202us) client_negotiation.cc:769] Sending connection context
0808 03:52:03.380324 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 03:52:03.380629 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":374,"threads_started":1}
I20260808 03:52:03.381135 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:03.367458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:35109 (local address 127.26.146.190:42201)
0808 03:52:03.367629 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:03.367635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:03.368275 (+ 640us) server_negotiation.cc:407] Connection header received
0808 03:52:03.368450 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:03.368456 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:03.368543 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:03.368649 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:03.369498 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:03.370657 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:03.371942 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:03.372462 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:03.372581 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:03.375288 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:03.375296 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:03.375301 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:03.375343 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:03.378149 (+ 2806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:03.379010 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:03.379016 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:03.379020 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:03.379148 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:03.379503 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:03.379508 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:03.379512 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:03.379790 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:03.379956 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:03.380462 (+ 506us) server_negotiation.cc:299] Negotiation successful
0808 03:52:03.380727 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 03:52:03.392545 5906 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:40873
I20260808 03:52:03.392568 6099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:40873 every 8 connection(s)
I20260808 03:52:03.393713 5906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:52:03.395579 6100 heartbeater.cc:347] Connected to a master server at 127.26.146.190:42201
I20260808 03:52:03.395793 6100 heartbeater.cc:464] Registering TS with master...
I20260808 03:52:03.396464 6100 heartbeater.cc:511] Master 127.26.146.190:42201 requested a full tablet report, sending...
I20260808 03:52:03.398366 5803 ts_manager.cc:194] Registered new tserver with Master: a566c5a4d4494120ac546c23153d5e04 (127.26.146.129:40873)
I20260808 03:52:03.399919 5803 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.146.129@KRBTEST.COM'} at 127.26.146.129:55493
I20260808 03:52:03.400159 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 5906
I20260808 03:52:03.400357 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:52:03.401086 27210 external_mini_cluster.cc:1428] Setting key 74c2cde25278d05f73d09e8a3f556a21
WARNING: no policy specified for oryx/127.26.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.130.keytab.
Entry for principal oryx/127.26.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.130.keytab.
I20260808 03:52:03.455295 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:42201
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.130.keytab
--principal=oryx/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:46519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:52:03.781164 6106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:03.781457 6106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:03.781603 6106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:03.788829 6106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:03.788940 6106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:03.789072 6106 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 03:52:03.789186 6106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:52:03.798817 6106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:46519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.130.keytab
--principal=oryx/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.146.190:42201
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:03.800307 6106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:03.801700 6106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:03.815341 6112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:03.815295 6117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:03.815373 6113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:03.816872 6106 server_base.cc:1034] running on GCE node
I20260808 03:52:03.817515 6106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:03.818465 6106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:03.819679 6106 hybrid_clock.cc:648] HybridClock initialized: now 1786161123819608 us; error 45 us; skew 500 ppm
I20260808 03:52:03.820014 6106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:52:03 dist-test-slave-jlzn krb5kdc[5740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161123, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:52:03.827033 6106 init.cc:377] Logged in from keytab as oryx/127.26.146.130@KRBTEST.COM (short username oryx)
I20260808 03:52:03.828615 6106 webserver.cc:460] Webserver started at http://127.26.146.130:33261/ using document root <none> and password file <none>
I20260808 03:52:03.829324 6106 fs_manager.cc:359] Metadata directory not provided
I20260808 03:52:03.829427 6106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:52:03.829751 6106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:52:03.832737 6106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "f10f8c8f4c674ec39f201d90d662bf42"
format_stamp: "Formatted at 2026-08-08 03:52:03 on dist-test-slave-jlzn"
server_key: "134372875827ef3ce3f54563444cd4a7"
server_key_iv: "dcd77da8b44693fd95bad7237e26c71b"
server_key_version: "encryptionkey@0"
I20260808 03:52:03.833518 6106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "f10f8c8f4c674ec39f201d90d662bf42"
format_stamp: "Formatted at 2026-08-08 03:52:03 on dist-test-slave-jlzn"
server_key: "134372875827ef3ce3f54563444cd4a7"
server_key_iv: "dcd77da8b44693fd95bad7237e26c71b"
server_key_version: "encryptionkey@0"
I20260808 03:52:03.838719 6106 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260808 03:52:03.842000 6133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:03.843258 6106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 03:52:03.843413 6106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "f10f8c8f4c674ec39f201d90d662bf42"
format_stamp: "Formatted at 2026-08-08 03:52:03 on dist-test-slave-jlzn"
server_key: "134372875827ef3ce3f54563444cd4a7"
server_key_iv: "dcd77da8b44693fd95bad7237e26c71b"
server_key_version: "encryptionkey@0"
I20260808 03:52:03.843619 6106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:52:03.858096 6106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:52:03.861997 6106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:52:03.862267 6106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:52:03.865125 6106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:52:03.865229 6106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:03.865375 6106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:52:03.865453 6106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 03:52:03 dist-test-slave-jlzn krb5kdc[5740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161123, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM
I20260808 03:52:03.887939 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:03.865676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:40119 (local address 127.26.146.190:42201)
0808 03:52:03.865897 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:03.865904 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:03.867420 (+ 1516us) server_negotiation.cc:407] Connection header received
0808 03:52:03.869514 (+ 2094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:03.869520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:03.869602 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:03.869699 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:03.871878 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:03.873103 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:03.875382 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:03.876005 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:03.876126 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:03.880788 (+ 4662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:03.880796 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:03.880803 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:03.880877 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:03.884256 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:03.885121 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:03.885127 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:03.885130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:03.885299 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:03.885885 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:03.885891 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:03.885894 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:03.886240 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:03.886397 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:03.887459 (+ 1062us) server_negotiation.cc:299] Negotiation successful
0808 03:52:03.887634 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260808 03:52:03.889675 6150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:03.866355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:42201 (local address 127.26.146.130:40119)
0808 03:52:03.867246 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:03.867288 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:03.869237 (+ 1949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:03.870002 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:03.870015 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:03.870806 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:03.871643 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:03.871675 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:03.873290 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:03.873295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:03.875228 (+ 1933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:03.875241 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:03.876269 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:03.876277 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:03.876560 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:03.877368 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:03.877403 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:03.880544 (+ 3141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:52:03.884406 (+ 3862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:03.884416 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:03.884438 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:03.884996 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:03.885459 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:03.885465 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:03.885469 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:03.885750 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:03.886418 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:52:03.886427 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:52:03.886973 (+ 546us) client_negotiation.cc:769] Sending connection context
0808 03:52:03.887349 (+ 376us) client_negotiation.cc:240] Negotiation successful
0808 03:52:03.887955 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":391,"threads_started":1}
I20260808 03:52:03.912791 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:03.895651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:42201 (local address 127.26.146.130:55421)
0808 03:52:03.900601 (+ 4950us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:03.900624 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:03.900755 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:03.901065 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:03.901069 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:03.901531 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:03.901821 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:03.901830 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:03.902841 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:03.902847 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:03.903951 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:03.903967 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:03.905058 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:03.905067 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:03.905201 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:03.905771 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:03.905788 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:03.906526 (+ 738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:52:03.910128 (+ 3602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:03.910133 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:03.910137 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:03.910558 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:03.910941 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:03.910947 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:03.910951 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:03.911119 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:03.911906 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:52:03.911911 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:52:03.912091 (+ 180us) client_negotiation.cc:769] Sending connection context
0808 03:52:03.912210 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 03:52:03.912457 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4792,"thread_start_us":4576,"threads_started":1}
I20260808 03:52:03.913127 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:03.895453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:55421 (local address 127.26.146.190:42201)
0808 03:52:03.895665 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:03.895671 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:03.900711 (+ 5040us) server_negotiation.cc:407] Connection header received
0808 03:52:03.900871 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:03.900877 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:03.900939 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:03.901018 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:03.901968 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:03.902710 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:03.904179 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:03.904826 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:03.904960 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:03.906709 (+ 1749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:03.906716 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:03.906721 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:03.906756 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:03.910021 (+ 3265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:03.910676 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:03.910682 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:03.910686 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:03.910827 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:03.911261 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:03.911266 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:03.911270 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:03.911755 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:03.911949 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:03.912337 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 03:52:03.912580 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 03:52:03.922766 6106 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:35087
I20260808 03:52:03.922806 6298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:35087 every 8 connection(s)
I20260808 03:52:03.924031 6106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:52:03.926120 6300 heartbeater.cc:347] Connected to a master server at 127.26.146.190:42201
I20260808 03:52:03.926340 6300 heartbeater.cc:464] Registering TS with master...
I20260808 03:52:03.926999 6300 heartbeater.cc:511] Master 127.26.146.190:42201 requested a full tablet report, sending...
I20260808 03:52:03.927923 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 6106
I20260808 03:52:03.928117 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:52:03.928555 5803 ts_manager.cc:194] Registered new tserver with Master: f10f8c8f4c674ec39f201d90d662bf42 (127.26.146.130:35087)
I20260808 03:52:03.928637 27210 external_mini_cluster.cc:1428] Setting key 396958ad720dc516c9df6f496e66fe8d
I20260808 03:52:03.929517 5803 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.146.130@KRBTEST.COM'} at 127.26.146.130:40119
WARNING: no policy specified for oryx/127.26.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.131.keytab.
Entry for principal oryx/127.26.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.131.keytab.
I20260808 03:52:03.983358 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:42201
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.131.keytab
--principal=oryx/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:46519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:52:04.304073 6309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:04.304335 6309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:04.304474 6309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:04.311640 6309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:04.311745 6309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:04.311898 6309 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 03:52:04.312019 6309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:52:04.321532 6309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:46519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.131.keytab
--principal=oryx/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.146.190:42201
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:04.322849 6309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:04.324242 6309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:04.336905 6315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:04.336869 6319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:04.336961 6316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:04.337700 6309 server_base.cc:1034] running on GCE node
I20260808 03:52:04.338253 6309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:04.339097 6309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:04.340305 6309 hybrid_clock.cc:648] HybridClock initialized: now 1786161124340247 us; error 31 us; skew 500 ppm
I20260808 03:52:04.340588 6309 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:52:04 dist-test-slave-jlzn krb5kdc[5740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161124, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:52:04.343662 6309 init.cc:377] Logged in from keytab as oryx/127.26.146.131@KRBTEST.COM (short username oryx)
I20260808 03:52:04.345296 6309 webserver.cc:460] Webserver started at http://127.26.146.131:35251/ using document root <none> and password file <none>
I20260808 03:52:04.345948 6309 fs_manager.cc:359] Metadata directory not provided
I20260808 03:52:04.346045 6309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:52:04.346352 6309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:52:04.349278 6309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "6378fdf302634a1f8dbf853985952b1a"
format_stamp: "Formatted at 2026-08-08 03:52:04 on dist-test-slave-jlzn"
server_key: "05b376535c43af5fc1efd1648cfe7968"
server_key_iv: "73de67ecfac8c013bf3f71247f928a68"
server_key_version: "encryptionkey@0"
I20260808 03:52:04.350003 6309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "6378fdf302634a1f8dbf853985952b1a"
format_stamp: "Formatted at 2026-08-08 03:52:04 on dist-test-slave-jlzn"
server_key: "05b376535c43af5fc1efd1648cfe7968"
server_key_iv: "73de67ecfac8c013bf3f71247f928a68"
server_key_version: "encryptionkey@0"
I20260808 03:52:04.354897 6309 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 03:52:04.358208 6328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:04.359514 6309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 03:52:04.359663 6309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "6378fdf302634a1f8dbf853985952b1a"
format_stamp: "Formatted at 2026-08-08 03:52:04 on dist-test-slave-jlzn"
server_key: "05b376535c43af5fc1efd1648cfe7968"
server_key_iv: "73de67ecfac8c013bf3f71247f928a68"
server_key_version: "encryptionkey@0"
I20260808 03:52:04.359937 6309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:52:04.375543 6309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:52:04.379441 6309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:52:04.379683 6309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:52:04.382346 6309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:52:04.382432 6309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:04.382576 6309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:52:04.382692 6309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 08 03:52:04 dist-test-slave-jlzn krb5kdc[5740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161124, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM
I20260808 03:52:04.402628 6100 heartbeater.cc:503] Master 127.26.146.190:42201 was elected leader, sending a full tablet report...
I20260808 03:52:04.405707 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:04.382821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:37921 (local address 127.26.146.190:42201)
0808 03:52:04.383045 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:04.383052 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:04.384430 (+ 1378us) server_negotiation.cc:407] Connection header received
0808 03:52:04.386491 (+ 2061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:04.386499 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:04.386605 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:04.386718 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:04.388797 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.389830 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.392794 (+ 2964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.393434 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.393687 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:04.398617 (+ 4930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:04.398626 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:04.398631 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:04.398673 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:04.402225 (+ 3552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:04.403032 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:04.403037 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:04.403039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:04.403202 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:04.403780 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:04.403785 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:04.403788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:04.404183 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:04.404367 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:04.405113 (+ 746us) server_negotiation.cc:299] Negotiation successful
0808 03:52:04.405355 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 03:52:04.406665 6345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:04.383339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:42201 (local address 127.26.146.131:37921)
0808 03:52:04.384236 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:04.384288 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:04.386201 (+ 1913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:04.386955 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:04.386963 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:04.387637 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:04.388551 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:04.388581 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.390036 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.390045 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:04.392607 (+ 2562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:04.392625 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.393748 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.393756 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:04.394023 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:04.394885 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:04.394917 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:04.398282 (+ 3365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:52:04.402389 (+ 4107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:04.402398 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:04.402414 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:04.402900 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:04.403340 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:04.403346 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:04.403350 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:04.403624 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:04.404340 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:52:04.404348 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:52:04.404703 (+ 355us) client_negotiation.cc:769] Sending connection context
0808 03:52:04.404935 (+ 232us) client_negotiation.cc:240] Negotiation successful
0808 03:52:04.405456 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":382,"threads_started":1}
I20260808 03:52:04.426571 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:04.412634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:50817 (local address 127.26.146.190:42201)
0808 03:52:04.413401 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:04.413405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:04.413448 (+ 43us) server_negotiation.cc:407] Connection header received
0808 03:52:04.413673 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:04.413677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:04.413750 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:04.413827 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:04.414985 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.415698 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.417628 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.418028 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.418122 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:04.420541 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:04.420548 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:04.420553 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:04.420592 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:04.423647 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:04.424440 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:04.424445 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:04.424448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:04.424596 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:04.424991 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:04.424997 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:04.425000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:04.425276 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:04.425422 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:04.426006 (+ 584us) server_negotiation.cc:299] Negotiation successful
0808 03:52:04.426228 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663}
I20260808 03:52:04.426635 6418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:04.412682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:42201 (local address 127.26.146.131:50817)
0808 03:52:04.413363 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:04.413384 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:04.413543 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:04.413873 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:04.413878 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:04.414430 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:04.414812 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:04.414824 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.415844 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.415849 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:04.417473 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:04.417487 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.418205 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.418211 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:04.418416 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:04.419039 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:04.419062 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:04.420330 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:52:04.423774 (+ 3444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:04.423780 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:04.423784 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:04.424275 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:04.424722 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:04.424727 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:04.424731 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:04.424869 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:04.425501 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:52:04.425509 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:52:04.425717 (+ 208us) client_negotiation.cc:769] Sending connection context
0808 03:52:04.425845 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 03:52:04.426178 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":352,"threads_started":1}
I20260808 03:52:04.441058 6309 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:45717
I20260808 03:52:04.441080 6500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:45717 every 8 connection(s)
I20260808 03:52:04.442250 6309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:52:04.444221 6501 heartbeater.cc:347] Connected to a master server at 127.26.146.190:42201
I20260808 03:52:04.444437 6501 heartbeater.cc:464] Registering TS with master...
I20260808 03:52:04.445087 6501 heartbeater.cc:511] Master 127.26.146.190:42201 requested a full tablet report, sending...
I20260808 03:52:04.446455 5800 ts_manager.cc:194] Registered new tserver with Master: 6378fdf302634a1f8dbf853985952b1a (127.26.146.131:45717)
I20260808 03:52:04.446657 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 6309
I20260808 03:52:04.446828 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:52:04.447317 27210 external_mini_cluster.cc:1428] Setting key 2f995c7976698575ebc5fb4ea6d45342
I20260808 03:52:04.447499 5800 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.146.131@KRBTEST.COM'} at 127.26.146.131:37921
I20260808 03:52:04.450127 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 03:52:04 dist-test-slave-jlzn krb5kdc[5740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161124, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 03:52:04 dist-test-slave-jlzn krb5kdc[5740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:04 dist-test-slave-jlzn krb5kdc[5740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 03:52:04.505699 5939 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:04.495282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47654 (local address 127.26.146.190:42201)
0808 03:52:04.495520 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:04.495527 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:04.495628 (+ 101us) server_negotiation.cc:407] Connection header received
0808 03:52:04.495864 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:04.495868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:04.495935 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:04.496021 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:04.497399 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.498128 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.499421 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.499886 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.500005 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:04.505351 (+ 5346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47654: BlockingRecv error: recv got EOF from 127.0.0.1:47654 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 08 03:52:04 dist-test-slave-jlzn krb5kdc[5740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161124, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM
I20260808 03:52:04.521718 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:04.508999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47664 (local address 127.26.146.190:42201)
0808 03:52:04.509178 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:04.509184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:04.509504 (+ 320us) server_negotiation.cc:407] Connection header received
0808 03:52:04.509670 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:04.509675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:04.509749 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:04.509831 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:04.510934 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.511669 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.512899 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.513332 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.513442 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:04.516999 (+ 3557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:04.517004 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:04.517008 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:04.517042 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:04.519522 (+ 2480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:04.520154 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:04.520157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:04.520160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:04.520283 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:04.520593 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:04.520596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:04.520598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:04.520849 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:04.521077 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:04.521281 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 03:52:04.521450 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 03:52:04.525484 5800 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:52:04.532323 5800 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:52:04.569751 6532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:04.543783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52108 (local address 127.26.146.130:35087)
0808 03:52:04.561548 (+ 17765us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:04.561556 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:04.561579 (+ 23us) server_negotiation.cc:407] Connection header received
0808 03:52:04.561651 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:04.561656 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:04.561817 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:04.561980 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:52:04.562850 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.564070 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.566846 (+ 2776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.568289 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.568457 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:04.568705 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:04.569310 (+ 605us) server_negotiation.cc:299] Negotiation successful
0808 03:52:04.569475 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17630,"thread_start_us":17474,"threads_started":1}
I20260808 03:52:04.569887 6533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:04.543705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:40873 (local address 127.0.0.1:34554)
0808 03:52:04.557313 (+ 13608us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:04.557329 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:04.557426 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:04.562081 (+ 4655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:04.562085 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:04.562109 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:52:04.562712 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:04.562727 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.564992 (+ 2265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.564996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:04.567161 (+ 2165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:04.567176 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.568691 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.568698 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:04.568950 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:04.569008 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 03:52:04.569130 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 03:52:04.569312 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13485,"mutex_wait_us":64,"thread_start_us":17888,"threads_started":1}
I20260808 03:52:04.570171 6529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:04.543553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:35087 (local address 127.0.0.1:52108)
0808 03:52:04.556480 (+ 12927us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:04.556528 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:04.556724 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:04.562024 (+ 5300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:04.562031 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:04.562061 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:52:04.562498 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:04.562516 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.564224 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.564229 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:04.566630 (+ 2401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:04.566648 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.568866 (+ 2218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.568873 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:04.569093 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:04.569126 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 03:52:04.569224 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 03:52:04.569365 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12585,"mutex_wait_us":186,"spinlock_wait_cycles":2944,"thread_start_us":12432,"threads_started":1}
I20260808 03:52:04.570500 6536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:04.544823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39942 (local address 127.26.146.131:45717)
0808 03:52:04.560366 (+ 15543us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:04.560377 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:04.561957 (+ 1580us) server_negotiation.cc:407] Connection header received
0808 03:52:04.562154 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:04.562162 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:04.562364 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:04.562578 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:52:04.563212 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.565009 (+ 1797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.567367 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.568885 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.569103 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:04.569267 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:04.570024 (+ 757us) server_negotiation.cc:299] Negotiation successful
0808 03:52:04.570181 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15355,"thread_start_us":15374,"threads_started":1}
I20260808 03:52:04.571007 6530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:04.544118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:45717 (local address 127.0.0.1:39942)
0808 03:52:04.561834 (+ 17716us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:04.561848 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:04.561961 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:04.562569 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:04.562573 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:04.562601 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:52:04.562998 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:04.563010 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.565174 (+ 2164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.565183 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:04.567182 (+ 1999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:04.567197 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.569411 (+ 2214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.569417 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:04.569615 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:04.569648 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 03:52:04.569740 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 03:52:04.569867 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17609,"mutex_wait_us":62,"thread_start_us":12908,"threads_started":1}
I20260808 03:52:04.571084 6535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:04.544197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34554 (local address 127.26.146.129:40873)
0808 03:52:04.560545 (+ 16348us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:04.560555 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:04.560591 (+ 36us) server_negotiation.cc:407] Connection header received
0808 03:52:04.560701 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:04.560710 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:04.560996 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:04.562907 (+ 1911us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:52:04.563117 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.564835 (+ 1718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.567327 (+ 2492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.568488 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.570231 (+ 1743us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:04.570495 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:04.570563 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 03:52:04.570814 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15926,"thread_start_us":15786,"threads_started":1}
I20260808 03:52:04.573835 6209 tablet_service.cc:1467] Processing CreateTablet for tablet 7fa16cdca2d0474987d250e4ab8c2fc2 (DEFAULT_TABLE table=test-table [id=36f757dbcb494689bb8ee50e5cb505f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:52:04.574362 6401 tablet_service.cc:1467] Processing CreateTablet for tablet 7fa16cdca2d0474987d250e4ab8c2fc2 (DEFAULT_TABLE table=test-table [id=36f757dbcb494689bb8ee50e5cb505f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:52:04.574362 6003 tablet_service.cc:1467] Processing CreateTablet for tablet 7fa16cdca2d0474987d250e4ab8c2fc2 (DEFAULT_TABLE table=test-table [id=36f757dbcb494689bb8ee50e5cb505f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:52:04.575232 6209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fa16cdca2d0474987d250e4ab8c2fc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:52:04.575645 6003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fa16cdca2d0474987d250e4ab8c2fc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:52:04.575657 6401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fa16cdca2d0474987d250e4ab8c2fc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:52:04.586189 6538 tablet_bootstrap.cc:492] T 7fa16cdca2d0474987d250e4ab8c2fc2 P f10f8c8f4c674ec39f201d90d662bf42: Bootstrap starting.
I20260808 03:52:04.589563 6539 tablet_bootstrap.cc:492] T 7fa16cdca2d0474987d250e4ab8c2fc2 P 6378fdf302634a1f8dbf853985952b1a: Bootstrap starting.
I20260808 03:52:04.590236 6538 tablet_bootstrap.cc:654] T 7fa16cdca2d0474987d250e4ab8c2fc2 P f10f8c8f4c674ec39f201d90d662bf42: Neither blocks nor log segments found. Creating new log.
I20260808 03:52:04.590929 6540 tablet_bootstrap.cc:492] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04: Bootstrap starting.
I20260808 03:52:04.591714 6538 log.cc:824] T 7fa16cdca2d0474987d250e4ab8c2fc2 P f10f8c8f4c674ec39f201d90d662bf42: Log is configured to *not* fsync() on all Append() calls
I20260808 03:52:04.592948 6539 tablet_bootstrap.cc:654] T 7fa16cdca2d0474987d250e4ab8c2fc2 P 6378fdf302634a1f8dbf853985952b1a: Neither blocks nor log segments found. Creating new log.
I20260808 03:52:04.593822 6540 tablet_bootstrap.cc:654] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04: Neither blocks nor log segments found. Creating new log.
I20260808 03:52:04.594172 6539 log.cc:824] T 7fa16cdca2d0474987d250e4ab8c2fc2 P 6378fdf302634a1f8dbf853985952b1a: Log is configured to *not* fsync() on all Append() calls
I20260808 03:52:04.594812 6540 log.cc:824] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04: Log is configured to *not* fsync() on all Append() calls
I20260808 03:52:04.595060 6538 tablet_bootstrap.cc:492] T 7fa16cdca2d0474987d250e4ab8c2fc2 P f10f8c8f4c674ec39f201d90d662bf42: No bootstrap required, opened a new log
I20260808 03:52:04.595317 6538 ts_tablet_manager.cc:1397] T 7fa16cdca2d0474987d250e4ab8c2fc2 P f10f8c8f4c674ec39f201d90d662bf42: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260808 03:52:04.596937 6539 tablet_bootstrap.cc:492] T 7fa16cdca2d0474987d250e4ab8c2fc2 P 6378fdf302634a1f8dbf853985952b1a: No bootstrap required, opened a new log
I20260808 03:52:04.597254 6539 ts_tablet_manager.cc:1397] T 7fa16cdca2d0474987d250e4ab8c2fc2 P 6378fdf302634a1f8dbf853985952b1a: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260808 03:52:04.597982 6540 tablet_bootstrap.cc:492] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04: No bootstrap required, opened a new log
I20260808 03:52:04.598234 6540 ts_tablet_manager.cc:1397] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260808 03:52:04.603173 6538 raft_consensus.cc:348] T 7fa16cdca2d0474987d250e4ab8c2fc2 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:04.603680 6538 raft_consensus.cc:374] T 7fa16cdca2d0474987d250e4ab8c2fc2 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:52:04.603832 6538 raft_consensus.cc:729] T 7fa16cdca2d0474987d250e4ab8c2fc2 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f10f8c8f4c674ec39f201d90d662bf42, State: Initialized, Role: FOLLOWER
I20260808 03:52:04.604524 6538 consensus_queue.cc:260] T 7fa16cdca2d0474987d250e4ab8c2fc2 P f10f8c8f4c674ec39f201d90d662bf42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:04.605554 6300 heartbeater.cc:503] Master 127.26.146.190:42201 was elected leader, sending a full tablet report...
I20260808 03:52:04.606237 6538 ts_tablet_manager.cc:1428] T 7fa16cdca2d0474987d250e4ab8c2fc2 P f10f8c8f4c674ec39f201d90d662bf42: Time spent starting tablet: real 0.011s user 0.006s sys 0.003s
I20260808 03:52:04.607342 6539 raft_consensus.cc:348] T 7fa16cdca2d0474987d250e4ab8c2fc2 P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:04.607856 6539 raft_consensus.cc:374] T 7fa16cdca2d0474987d250e4ab8c2fc2 P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:52:04.608006 6539 raft_consensus.cc:729] T 7fa16cdca2d0474987d250e4ab8c2fc2 P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6378fdf302634a1f8dbf853985952b1a, State: Initialized, Role: FOLLOWER
I20260808 03:52:04.608439 6540 raft_consensus.cc:348] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:04.608850 6540 raft_consensus.cc:374] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:52:04.608726 6539 consensus_queue.cc:260] T 7fa16cdca2d0474987d250e4ab8c2fc2 P 6378fdf302634a1f8dbf853985952b1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:04.608954 6540 raft_consensus.cc:729] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a566c5a4d4494120ac546c23153d5e04, State: Initialized, Role: FOLLOWER
I20260808 03:52:04.609591 6540 consensus_queue.cc:260] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:04.610134 6501 heartbeater.cc:503] Master 127.26.146.190:42201 was elected leader, sending a full tablet report...
I20260808 03:52:04.610745 6539 ts_tablet_manager.cc:1428] T 7fa16cdca2d0474987d250e4ab8c2fc2 P 6378fdf302634a1f8dbf853985952b1a: Time spent starting tablet: real 0.013s user 0.007s sys 0.005s
I20260808 03:52:04.612344 6540 ts_tablet_manager.cc:1428] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
W20260808 03:52:04.645890 6103 tablet.cc:2369] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:52:04.676019 6302 tablet.cc:2369] T 7fa16cdca2d0474987d250e4ab8c2fc2 P f10f8c8f4c674ec39f201d90d662bf42: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:52:04.691839 6547 raft_consensus.cc:482] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:52:04.692162 6547 raft_consensus.cc:504] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
W20260808 03:52:04.693717 6503 tablet.cc:2369] T 7fa16cdca2d0474987d250e4ab8c2fc2 P 6378fdf302634a1f8dbf853985952b1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:52:04.695267 6547 leader_election.cc:288] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f10f8c8f4c674ec39f201d90d662bf42 (127.26.146.130:35087), 6378fdf302634a1f8dbf853985952b1a (127.26.146.131:45717)
I20260808 03:52:04.702836 6536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:04.694518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:46311 (local address 127.26.146.131:45717)
0808 03:52:04.694766 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:04.694785 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:04.695627 (+ 842us) server_negotiation.cc:407] Connection header received
0808 03:52:04.695774 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:04.695777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:04.695869 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:04.695954 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:52:04.696724 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.697740 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.700566 (+ 2826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.701756 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.701918 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:04.701994 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:04.702452 (+ 458us) server_negotiation.cc:299] Negotiation successful
0808 03:52:04.702592 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 03:52:04.703024 6549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:04.694217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:35087 (local address 127.26.146.129:33751)
0808 03:52:04.695363 (+ 1146us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:04.695389 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:04.695552 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:04.695951 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:04.695956 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:04.695980 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:52:04.696436 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:04.696449 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.697869 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.697874 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:04.700410 (+ 2536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:04.700419 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.702011 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.702017 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:04.702242 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:04.702323 (+ 81us) client_negotiation.cc:769] Sending connection context
0808 03:52:04.702407 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 03:52:04.702574 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":970,"mutex_wait_us":203,"thread_start_us":1010,"threads_started":1}
I20260808 03:52:04.703081 6548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:04.694298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:45717 (local address 127.26.146.129:46311)
0808 03:52:04.695505 (+ 1207us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:04.695523 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:04.695650 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:04.696109 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:04.696117 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:04.696195 (+ 78us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:52:04.696568 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:04.696578 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.697871 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.697875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:04.700402 (+ 2527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:04.700419 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.701974 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.701980 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:04.702244 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:04.702335 (+ 91us) client_negotiation.cc:769] Sending connection context
0808 03:52:04.702412 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 03:52:04.702574 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1089,"mutex_wait_us":205,"spinlock_wait_cycles":25344,"thread_start_us":635,"threads_started":1}
I20260808 03:52:04.703977 6532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:04.694791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:33751 (local address 127.26.146.130:35087)
0808 03:52:04.694996 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:04.695002 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:04.695509 (+ 507us) server_negotiation.cc:407] Connection header received
0808 03:52:04.695721 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:04.695726 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:04.695829 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:04.695918 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:52:04.696592 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.697736 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.700566 (+ 2830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.701756 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.701918 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:04.703532 (+ 1614us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:04.703582 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 03:52:04.703706 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 03:52:04.704209 6434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fa16cdca2d0474987d250e4ab8c2fc2" candidate_uuid: "a566c5a4d4494120ac546c23153d5e04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6378fdf302634a1f8dbf853985952b1a" is_pre_election: true
I20260808 03:52:04.704793 6434 raft_consensus.cc:2454] T 7fa16cdca2d0474987d250e4ab8c2fc2 P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a566c5a4d4494120ac546c23153d5e04 in term 0.
I20260808 03:52:04.704870 6237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fa16cdca2d0474987d250e4ab8c2fc2" candidate_uuid: "a566c5a4d4494120ac546c23153d5e04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f10f8c8f4c674ec39f201d90d662bf42" is_pre_election: true
I20260808 03:52:04.705366 6237 raft_consensus.cc:2454] T 7fa16cdca2d0474987d250e4ab8c2fc2 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a566c5a4d4494120ac546c23153d5e04 in term 0.
I20260808 03:52:04.705536 5930 leader_election.cc:302] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6378fdf302634a1f8dbf853985952b1a, a566c5a4d4494120ac546c23153d5e04; no voters:
I20260808 03:52:04.708500 6547 raft_consensus.cc:2781] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:52:04.708714 6547 raft_consensus.cc:482] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:52:04.708828 6547 raft_consensus.cc:3037] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:52:04.711980 6547 raft_consensus.cc:504] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:04.712638 6547 leader_election.cc:288] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04 [CANDIDATE]: Term 1 election: Requested vote from peers f10f8c8f4c674ec39f201d90d662bf42 (127.26.146.130:35087), 6378fdf302634a1f8dbf853985952b1a (127.26.146.131:45717)
I20260808 03:52:04.713060 6434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fa16cdca2d0474987d250e4ab8c2fc2" candidate_uuid: "a566c5a4d4494120ac546c23153d5e04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6378fdf302634a1f8dbf853985952b1a"
I20260808 03:52:04.713121 6237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fa16cdca2d0474987d250e4ab8c2fc2" candidate_uuid: "a566c5a4d4494120ac546c23153d5e04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f10f8c8f4c674ec39f201d90d662bf42"
I20260808 03:52:04.713243 6434 raft_consensus.cc:3037] T 7fa16cdca2d0474987d250e4ab8c2fc2 P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:52:04.713325 6237 raft_consensus.cc:3037] T 7fa16cdca2d0474987d250e4ab8c2fc2 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:52:04.715318 6434 raft_consensus.cc:2454] T 7fa16cdca2d0474987d250e4ab8c2fc2 P 6378fdf302634a1f8dbf853985952b1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a566c5a4d4494120ac546c23153d5e04 in term 1.
I20260808 03:52:04.715682 5930 leader_election.cc:302] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6378fdf302634a1f8dbf853985952b1a, a566c5a4d4494120ac546c23153d5e04; no voters:
I20260808 03:52:04.715983 6547 raft_consensus.cc:2781] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:52:04.716420 6237 raft_consensus.cc:2454] T 7fa16cdca2d0474987d250e4ab8c2fc2 P f10f8c8f4c674ec39f201d90d662bf42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a566c5a4d4494120ac546c23153d5e04 in term 1.
I20260808 03:52:04.719379 6547 raft_consensus.cc:686] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04 [term 1 LEADER]: Becoming Leader. State: Replica: a566c5a4d4494120ac546c23153d5e04, State: Running, Role: LEADER
I20260808 03:52:04.719951 6547 consensus_queue.cc:237] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:04.724355 5800 catalog_manager.cc:5495] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04 reported cstate change: term changed from 0 to 1, leader changed from <none> to a566c5a4d4494120ac546c23153d5e04 (127.26.146.129). New cstate: current_term: 1 leader_uuid: "a566c5a4d4494120ac546c23153d5e04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } health_report { overall_health: UNKNOWN } } }
I20260808 03:52:04.758081 6535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:04.753353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34564 (local address 127.26.146.129:40873)
0808 03:52:04.753541 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:04.753546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:04.753572 (+ 26us) server_negotiation.cc:407] Connection header received
0808 03:52:04.753701 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:04.753705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:04.753797 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:04.753928 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:52:04.754200 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.754973 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.756212 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.756624 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.756716 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:04.757104 (+ 388us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:52:04.757649 (+ 545us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:52:04.757759 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:04.757796 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 03:52:04.757860 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 03:52:04.766516 6434 raft_consensus.cc:1261] T 7fa16cdca2d0474987d250e4ab8c2fc2 P 6378fdf302634a1f8dbf853985952b1a [term 1 FOLLOWER]: Refusing update from remote peer a566c5a4d4494120ac546c23153d5e04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:52:04.766521 6237 raft_consensus.cc:1261] T 7fa16cdca2d0474987d250e4ab8c2fc2 P f10f8c8f4c674ec39f201d90d662bf42 [term 1 FOLLOWER]: Refusing update from remote peer a566c5a4d4494120ac546c23153d5e04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:52:04.767674 6550 consensus_queue.cc:1035] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:52:04.768142 6547 consensus_queue.cc:1035] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:52:04.794806 5800 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:52:04.795027 5800 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:52:04.798585 5800 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47: Sending DeleteTablet for 3 replicas of tablet 7fa16cdca2d0474987d250e4ab8c2fc2
I20260808 03:52:04.799661 6003 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fa16cdca2d0474987d250e4ab8c2fc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:52:04 UTC) from {username='oryx', principal='oryx/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:34554
I20260808 03:52:04.799760 6401 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fa16cdca2d0474987d250e4ab8c2fc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:52:04 UTC) from {username='oryx', principal='oryx/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:39942
I20260808 03:52:04.800999 6209 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fa16cdca2d0474987d250e4ab8c2fc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:52:04 UTC) from {username='oryx', principal='oryx/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:52108
I20260808 03:52:04.801035 5800 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:52:04.801659 5800 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:52:04.809410 6001 tablet_service.cc:1467] Processing CreateTablet for tablet f95041d155fe404384ac334e07fc57ac (DEFAULT_TABLE table=test-table [id=16f303951d964e71b13248493e2547a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:52:04.809572 6399 tablet_service.cc:1467] Processing CreateTablet for tablet f95041d155fe404384ac334e07fc57ac (DEFAULT_TABLE table=test-table [id=16f303951d964e71b13248493e2547a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:52:04.809901 6208 tablet_service.cc:1467] Processing CreateTablet for tablet f95041d155fe404384ac334e07fc57ac (DEFAULT_TABLE table=test-table [id=16f303951d964e71b13248493e2547a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:52:04.810034 6001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f95041d155fe404384ac334e07fc57ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:52:04.810176 6399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f95041d155fe404384ac334e07fc57ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:52:04.810444 6208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f95041d155fe404384ac334e07fc57ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:52:04.815833 6572 tablet_replica.cc:330] T 7fa16cdca2d0474987d250e4ab8c2fc2 P f10f8c8f4c674ec39f201d90d662bf42: stopping tablet replica
I20260808 03:52:04.816474 6572 raft_consensus.cc:2229] T 7fa16cdca2d0474987d250e4ab8c2fc2 P f10f8c8f4c674ec39f201d90d662bf42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:52:04.816642 6540 tablet_bootstrap.cc:492] T f95041d155fe404384ac334e07fc57ac P a566c5a4d4494120ac546c23153d5e04: Bootstrap starting.
I20260808 03:52:04.817152 6572 raft_consensus.cc:2258] T 7fa16cdca2d0474987d250e4ab8c2fc2 P f10f8c8f4c674ec39f201d90d662bf42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:52:04.818445 6540 tablet_bootstrap.cc:654] T f95041d155fe404384ac334e07fc57ac P a566c5a4d4494120ac546c23153d5e04: Neither blocks nor log segments found. Creating new log.
I20260808 03:52:04.819757 6539 tablet_bootstrap.cc:492] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a: Bootstrap starting.
I20260808 03:52:04.821597 6570 tablet_replica.cc:330] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04: stopping tablet replica
I20260808 03:52:04.821774 6572 ts_tablet_manager.cc:1905] T 7fa16cdca2d0474987d250e4ab8c2fc2 P f10f8c8f4c674ec39f201d90d662bf42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:52:04.822008 6539 tablet_bootstrap.cc:654] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a: Neither blocks nor log segments found. Creating new log.
I20260808 03:52:04.822021 6570 raft_consensus.cc:2229] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04 [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:52:04.822777 6570 raft_consensus.cc:2258] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:52:04.824792 6570 ts_tablet_manager.cc:1905] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:52:04.825415 6569 tablet_replica.cc:330] T 7fa16cdca2d0474987d250e4ab8c2fc2 P 6378fdf302634a1f8dbf853985952b1a: stopping tablet replica
I20260808 03:52:04.825853 6569 raft_consensus.cc:2229] T 7fa16cdca2d0474987d250e4ab8c2fc2 P 6378fdf302634a1f8dbf853985952b1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:52:04.826084 6538 tablet_bootstrap.cc:492] T f95041d155fe404384ac334e07fc57ac P f10f8c8f4c674ec39f201d90d662bf42: Bootstrap starting.
I20260808 03:52:04.827844 6538 tablet_bootstrap.cc:654] T f95041d155fe404384ac334e07fc57ac P f10f8c8f4c674ec39f201d90d662bf42: Neither blocks nor log segments found. Creating new log.
I20260808 03:52:04.825773 6540 tablet_bootstrap.cc:492] T f95041d155fe404384ac334e07fc57ac P a566c5a4d4494120ac546c23153d5e04: No bootstrap required, opened a new log
I20260808 03:52:04.829030 6540 ts_tablet_manager.cc:1397] T f95041d155fe404384ac334e07fc57ac P a566c5a4d4494120ac546c23153d5e04: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20260808 03:52:04.829898 6540 raft_consensus.cc:348] T f95041d155fe404384ac334e07fc57ac P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:04.830152 6540 raft_consensus.cc:374] T f95041d155fe404384ac334e07fc57ac P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:52:04.830323 6540 raft_consensus.cc:729] T f95041d155fe404384ac334e07fc57ac P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a566c5a4d4494120ac546c23153d5e04, State: Initialized, Role: FOLLOWER
I20260808 03:52:04.830679 6540 consensus_queue.cc:260] T f95041d155fe404384ac334e07fc57ac P a566c5a4d4494120ac546c23153d5e04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:04.831408 6540 ts_tablet_manager.cc:1428] T f95041d155fe404384ac334e07fc57ac P a566c5a4d4494120ac546c23153d5e04: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260808 03:52:04.836155 6572 ts_tablet_manager.cc:1918] T 7fa16cdca2d0474987d250e4ab8c2fc2 P f10f8c8f4c674ec39f201d90d662bf42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:52:04.836409 6572 log.cc:1196] T 7fa16cdca2d0474987d250e4ab8c2fc2 P f10f8c8f4c674ec39f201d90d662bf42: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/7fa16cdca2d0474987d250e4ab8c2fc2
I20260808 03:52:04.837051 6572 ts_tablet_manager.cc:1939] T 7fa16cdca2d0474987d250e4ab8c2fc2 P f10f8c8f4c674ec39f201d90d662bf42: Deleting consensus metadata
I20260808 03:52:04.838838 6570 ts_tablet_manager.cc:1918] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:52:04.839102 6570 log.cc:1196] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-0/wal/wals/7fa16cdca2d0474987d250e4ab8c2fc2
I20260808 03:52:04.839747 6570 ts_tablet_manager.cc:1939] T 7fa16cdca2d0474987d250e4ab8c2fc2 P a566c5a4d4494120ac546c23153d5e04: Deleting consensus metadata
I20260808 03:52:04.840514 5783 catalog_manager.cc:4841] TS f10f8c8f4c674ec39f201d90d662bf42 (127.26.146.130:35087): tablet 7fa16cdca2d0474987d250e4ab8c2fc2 (table test-table [id=36f757dbcb494689bb8ee50e5cb505f7]) successfully deleted
I20260808 03:52:04.832504 6539 tablet_bootstrap.cc:492] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a: No bootstrap required, opened a new log
I20260808 03:52:04.841512 6539 ts_tablet_manager.cc:1397] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.008s
I20260808 03:52:04.841504 5780 catalog_manager.cc:4841] TS a566c5a4d4494120ac546c23153d5e04 (127.26.146.129:40873): tablet 7fa16cdca2d0474987d250e4ab8c2fc2 (table test-table [id=36f757dbcb494689bb8ee50e5cb505f7]) successfully deleted
I20260808 03:52:04.842264 6539 raft_consensus.cc:348] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:04.842433 6539 raft_consensus.cc:374] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:52:04.842489 6539 raft_consensus.cc:729] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6378fdf302634a1f8dbf853985952b1a, State: Initialized, Role: FOLLOWER
I20260808 03:52:04.842744 6539 consensus_queue.cc:260] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:04.843333 6539 ts_tablet_manager.cc:1428] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 03:52:04.844719 6569 raft_consensus.cc:2258] T 7fa16cdca2d0474987d250e4ab8c2fc2 P 6378fdf302634a1f8dbf853985952b1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:52:04.847021 6538 tablet_bootstrap.cc:492] T f95041d155fe404384ac334e07fc57ac P f10f8c8f4c674ec39f201d90d662bf42: No bootstrap required, opened a new log
I20260808 03:52:04.847231 6538 ts_tablet_manager.cc:1397] T f95041d155fe404384ac334e07fc57ac P f10f8c8f4c674ec39f201d90d662bf42: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.012s
I20260808 03:52:04.847360 6569 ts_tablet_manager.cc:1905] T 7fa16cdca2d0474987d250e4ab8c2fc2 P 6378fdf302634a1f8dbf853985952b1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:52:04.848039 6538 raft_consensus.cc:348] T f95041d155fe404384ac334e07fc57ac P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:04.848263 6538 raft_consensus.cc:374] T f95041d155fe404384ac334e07fc57ac P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:52:04.848364 6538 raft_consensus.cc:729] T f95041d155fe404384ac334e07fc57ac P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f10f8c8f4c674ec39f201d90d662bf42, State: Initialized, Role: FOLLOWER
I20260808 03:52:04.848635 6538 consensus_queue.cc:260] T f95041d155fe404384ac334e07fc57ac P f10f8c8f4c674ec39f201d90d662bf42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:04.849234 6538 ts_tablet_manager.cc:1428] T f95041d155fe404384ac334e07fc57ac P f10f8c8f4c674ec39f201d90d662bf42: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 03:52:04.854272 6569 ts_tablet_manager.cc:1918] T 7fa16cdca2d0474987d250e4ab8c2fc2 P 6378fdf302634a1f8dbf853985952b1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:52:04.854446 6569 log.cc:1196] T 7fa16cdca2d0474987d250e4ab8c2fc2 P 6378fdf302634a1f8dbf853985952b1a: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/7fa16cdca2d0474987d250e4ab8c2fc2
I20260808 03:52:04.854979 6569 ts_tablet_manager.cc:1939] T 7fa16cdca2d0474987d250e4ab8c2fc2 P 6378fdf302634a1f8dbf853985952b1a: Deleting consensus metadata
I20260808 03:52:04.856804 5779 catalog_manager.cc:4841] TS 6378fdf302634a1f8dbf853985952b1a (127.26.146.131:45717): tablet 7fa16cdca2d0474987d250e4ab8c2fc2 (table test-table [id=36f757dbcb494689bb8ee50e5cb505f7]) successfully deleted
I20260808 03:52:04.862071 6546 raft_consensus.cc:482] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:52:04.862237 6546 raft_consensus.cc:504] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:04.863778 6546 leader_election.cc:288] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a566c5a4d4494120ac546c23153d5e04 (127.26.146.129:40873), f10f8c8f4c674ec39f201d90d662bf42 (127.26.146.130:35087)
I20260808 03:52:04.872871 6345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:04.864665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:35087 (local address 127.26.146.131:47537)
0808 03:52:04.865057 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:04.865104 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:04.865390 (+ 286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:04.866050 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:04.866058 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:04.866087 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:52:04.866624 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:04.866640 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.868627 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.868632 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:04.870687 (+ 2055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:04.870707 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.872138 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.872144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:04.872351 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:04.872376 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 03:52:04.872459 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 03:52:04.872595 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246}
I20260808 03:52:04.872890 6532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:04.864304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:47537 (local address 127.26.146.130:35087)
0808 03:52:04.864543 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:04.864551 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:04.865212 (+ 661us) server_negotiation.cc:407] Connection header received
0808 03:52:04.865629 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:04.865636 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:04.865751 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:04.865975 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:52:04.866871 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.868411 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.870888 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.871980 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.872136 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:04.872211 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:04.872506 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 03:52:04.872627 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 03:52:04.873638 6237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f95041d155fe404384ac334e07fc57ac" candidate_uuid: "6378fdf302634a1f8dbf853985952b1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f10f8c8f4c674ec39f201d90d662bf42" is_pre_election: true
I20260808 03:52:04.873875 6237 raft_consensus.cc:2454] T f95041d155fe404384ac334e07fc57ac P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6378fdf302634a1f8dbf853985952b1a in term 0.
I20260808 03:52:04.874398 6335 leader_election.cc:302] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6378fdf302634a1f8dbf853985952b1a, f10f8c8f4c674ec39f201d90d662bf42; no voters:
I20260808 03:52:04.874925 6546 raft_consensus.cc:2781] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:52:04.875013 6580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:04.864849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:40873 (local address 127.26.146.131:51803)
0808 03:52:04.867786 (+ 2937us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:04.867825 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:04.868014 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:04.868525 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:04.868530 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:04.868556 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:52:04.868986 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:04.868998 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.870543 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.870549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:04.872566 (+ 2017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:04.872581 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.874184 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.874191 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:04.874378 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:04.874415 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 03:52:04.874508 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 03:52:04.874633 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2798,"mutex_wait_us":137,"thread_start_us":6353,"threads_started":1}
I20260808 03:52:04.875036 6535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:04.864548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:51803 (local address 127.26.146.129:40873)
0808 03:52:04.864716 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:04.864721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:04.867909 (+ 3188us) server_negotiation.cc:407] Connection header received
0808 03:52:04.868176 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:04.868181 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:04.868261 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:04.868325 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:52:04.869192 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.870367 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.872765 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.873972 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.874135 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:04.874208 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:04.874567 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 03:52:04.874711 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 03:52:04.875056 6546 raft_consensus.cc:482] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:52:04.875145 6546 raft_consensus.cc:3037] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:52:04.876049 6040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f95041d155fe404384ac334e07fc57ac" candidate_uuid: "6378fdf302634a1f8dbf853985952b1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a566c5a4d4494120ac546c23153d5e04" is_pre_election: true
I20260808 03:52:04.876540 6040 raft_consensus.cc:2454] T f95041d155fe404384ac334e07fc57ac P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6378fdf302634a1f8dbf853985952b1a in term 0.
I20260808 03:52:04.878013 6546 raft_consensus.cc:504] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:04.878908 6546 leader_election.cc:288] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a [CANDIDATE]: Term 1 election: Requested vote from peers a566c5a4d4494120ac546c23153d5e04 (127.26.146.129:40873), f10f8c8f4c674ec39f201d90d662bf42 (127.26.146.130:35087)
I20260808 03:52:04.879051 6040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f95041d155fe404384ac334e07fc57ac" candidate_uuid: "6378fdf302634a1f8dbf853985952b1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a566c5a4d4494120ac546c23153d5e04"
I20260808 03:52:04.879062 6237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f95041d155fe404384ac334e07fc57ac" candidate_uuid: "6378fdf302634a1f8dbf853985952b1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f10f8c8f4c674ec39f201d90d662bf42"
I20260808 03:52:04.879266 6040 raft_consensus.cc:3037] T f95041d155fe404384ac334e07fc57ac P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:52:04.879288 6237 raft_consensus.cc:3037] T f95041d155fe404384ac334e07fc57ac P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:52:04.881644 6040 raft_consensus.cc:2454] T f95041d155fe404384ac334e07fc57ac P a566c5a4d4494120ac546c23153d5e04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6378fdf302634a1f8dbf853985952b1a in term 1.
I20260808 03:52:04.882036 6331 leader_election.cc:302] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6378fdf302634a1f8dbf853985952b1a, a566c5a4d4494120ac546c23153d5e04; no voters:
I20260808 03:52:04.882187 6237 raft_consensus.cc:2454] T f95041d155fe404384ac334e07fc57ac P f10f8c8f4c674ec39f201d90d662bf42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6378fdf302634a1f8dbf853985952b1a in term 1.
I20260808 03:52:04.882308 6546 raft_consensus.cc:2781] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:52:04.889555 6546 raft_consensus.cc:686] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a [term 1 LEADER]: Becoming Leader. State: Replica: 6378fdf302634a1f8dbf853985952b1a, State: Running, Role: LEADER
I20260808 03:52:04.890106 6546 consensus_queue.cc:237] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:04.894043 5800 catalog_manager.cc:5495] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6378fdf302634a1f8dbf853985952b1a (127.26.146.131). New cstate: current_term: 1 leader_uuid: "6378fdf302634a1f8dbf853985952b1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } health_report { overall_health: HEALTHY } } }
Aug 08 03:52:04 dist-test-slave-jlzn krb5kdc[5740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161122, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.190@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM
I20260808 03:52:04.960856 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:04.946722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47676 (local address 127.26.146.190:42201)
0808 03:52:04.946959 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:04.946965 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:04.947627 (+ 662us) server_negotiation.cc:407] Connection header received
0808 03:52:04.947832 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:04.947837 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:04.947911 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:04.948023 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:04.949248 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.950038 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.951285 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.951717 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.951990 (+ 273us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:04.955616 (+ 3626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:04.955621 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:04.955624 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:04.955655 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:04.958156 (+ 2501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:04.958863 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:04.958868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:04.958872 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:04.958985 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:04.959335 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:04.959340 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:04.959343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:04.959632 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:04.959775 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:04.960248 (+ 473us) server_negotiation.cc:299] Negotiation successful
0808 03:52:04.960538 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":104,"server-negotiator.queue_time_us":103}
I20260808 03:52:04.960886 6592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:04.946812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:42201 (local address 127.0.0.1:47676)
0808 03:52:04.947501 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:04.947521 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:04.947672 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:04.948040 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:04.948044 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:04.948822 (+ 778us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:04.949095 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:04.949106 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.950174 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.950179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:04.951141 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:04.951154 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:04.952006 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:04.952012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:04.952242 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:04.952918 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:04.952940 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:04.955427 (+ 2487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:52:04.958287 (+ 2860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:04.958300 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:04.958308 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:04.958748 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:04.959100 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:04.959105 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:04.959109 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:04.959237 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:04.959773 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:52:04.959782 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:52:04.959987 (+ 205us) client_negotiation.cc:769] Sending connection context
0808 03:52:04.960129 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 03:52:04.960408 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":29,"thread_start_us":361,"threads_started":1}
I20260808 03:52:04.966063 5800 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:47676:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 03:52:04.967104 5800 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:52:04.974010 6003 tablet_service.cc:1467] Processing CreateTablet for tablet 6c0980067d44475b9fb43b9a4ab467d3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c4d652e048b04c96928c15f0f576dbd8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 03:52:04.974128 6208 tablet_service.cc:1467] Processing CreateTablet for tablet 6c0980067d44475b9fb43b9a4ab467d3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c4d652e048b04c96928c15f0f576dbd8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 03:52:04.974606 6208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c0980067d44475b9fb43b9a4ab467d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:52:04.974639 6003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c0980067d44475b9fb43b9a4ab467d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:52:04.979699 6538 tablet_bootstrap.cc:492] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42: Bootstrap starting.
I20260808 03:52:04.981254 6538 tablet_bootstrap.cc:654] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42: Neither blocks nor log segments found. Creating new log.
I20260808 03:52:04.982084 6540 tablet_bootstrap.cc:492] T 6c0980067d44475b9fb43b9a4ab467d3 P a566c5a4d4494120ac546c23153d5e04: Bootstrap starting.
I20260808 03:52:04.982581 6401 tablet_service.cc:1467] Processing CreateTablet for tablet 6c0980067d44475b9fb43b9a4ab467d3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c4d652e048b04c96928c15f0f576dbd8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 03:52:04.983161 6401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c0980067d44475b9fb43b9a4ab467d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:52:04.983999 6540 tablet_bootstrap.cc:654] T 6c0980067d44475b9fb43b9a4ab467d3 P a566c5a4d4494120ac546c23153d5e04: Neither blocks nor log segments found. Creating new log.
I20260808 03:52:04.990737 6539 tablet_bootstrap.cc:492] T 6c0980067d44475b9fb43b9a4ab467d3 P 6378fdf302634a1f8dbf853985952b1a: Bootstrap starting.
I20260808 03:52:04.991336 6538 tablet_bootstrap.cc:492] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42: No bootstrap required, opened a new log
I20260808 03:52:04.991587 6538 ts_tablet_manager.cc:1397] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.003s
I20260808 03:52:04.992415 6539 tablet_bootstrap.cc:654] T 6c0980067d44475b9fb43b9a4ab467d3 P 6378fdf302634a1f8dbf853985952b1a: Neither blocks nor log segments found. Creating new log.
I20260808 03:52:04.992416 6538 raft_consensus.cc:348] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:04.992699 6538 raft_consensus.cc:374] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:52:04.992835 6538 raft_consensus.cc:729] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f10f8c8f4c674ec39f201d90d662bf42, State: Initialized, Role: FOLLOWER
I20260808 03:52:04.993148 6538 consensus_queue.cc:260] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:04.993614 6545 tablet_replica.cc:439] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } } }
I20260808 03:52:04.993984 6545 tablet_replica.cc:442] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 03:52:04.994675 6539 tablet_bootstrap.cc:492] T 6c0980067d44475b9fb43b9a4ab467d3 P 6378fdf302634a1f8dbf853985952b1a: No bootstrap required, opened a new log
I20260808 03:52:04.994827 6540 tablet_bootstrap.cc:492] T 6c0980067d44475b9fb43b9a4ab467d3 P a566c5a4d4494120ac546c23153d5e04: No bootstrap required, opened a new log
I20260808 03:52:04.994848 6539 ts_tablet_manager.cc:1397] T 6c0980067d44475b9fb43b9a4ab467d3 P 6378fdf302634a1f8dbf853985952b1a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 03:52:04.995066 6540 ts_tablet_manager.cc:1397] T 6c0980067d44475b9fb43b9a4ab467d3 P a566c5a4d4494120ac546c23153d5e04: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.004s
I20260808 03:52:04.995591 6539 raft_consensus.cc:348] T 6c0980067d44475b9fb43b9a4ab467d3 P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:04.995853 6539 raft_consensus.cc:374] T 6c0980067d44475b9fb43b9a4ab467d3 P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:52:04.995950 6539 raft_consensus.cc:729] T 6c0980067d44475b9fb43b9a4ab467d3 P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6378fdf302634a1f8dbf853985952b1a, State: Initialized, Role: FOLLOWER
I20260808 03:52:04.996043 6540 raft_consensus.cc:348] T 6c0980067d44475b9fb43b9a4ab467d3 P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:04.996186 6539 consensus_queue.cc:260] T 6c0980067d44475b9fb43b9a4ab467d3 P 6378fdf302634a1f8dbf853985952b1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:04.996335 6540 raft_consensus.cc:374] T 6c0980067d44475b9fb43b9a4ab467d3 P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:52:04.996513 6540 raft_consensus.cc:729] T 6c0980067d44475b9fb43b9a4ab467d3 P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a566c5a4d4494120ac546c23153d5e04, State: Initialized, Role: FOLLOWER
I20260808 03:52:04.996724 6539 ts_tablet_manager.cc:1428] T 6c0980067d44475b9fb43b9a4ab467d3 P 6378fdf302634a1f8dbf853985952b1a: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260808 03:52:04.996587 6546 tablet_replica.cc:439] T 6c0980067d44475b9fb43b9a4ab467d3 P 6378fdf302634a1f8dbf853985952b1a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } } }
I20260808 03:52:04.996913 6546 tablet_replica.cc:442] T 6c0980067d44475b9fb43b9a4ab467d3 P 6378fdf302634a1f8dbf853985952b1a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 03:52:04.996907 6540 consensus_queue.cc:260] T 6c0980067d44475b9fb43b9a4ab467d3 P a566c5a4d4494120ac546c23153d5e04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:04.997568 6540 ts_tablet_manager.cc:1428] T 6c0980067d44475b9fb43b9a4ab467d3 P a566c5a4d4494120ac546c23153d5e04: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260808 03:52:04.997459 6550 tablet_replica.cc:439] T 6c0980067d44475b9fb43b9a4ab467d3 P a566c5a4d4494120ac546c23153d5e04: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } } }
I20260808 03:52:04.997807 6550 tablet_replica.cc:442] T 6c0980067d44475b9fb43b9a4ab467d3 P a566c5a4d4494120ac546c23153d5e04: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 03:52:04.998335 6538 ts_tablet_manager.cc:1428] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260808 03:52:05.039460 6545 raft_consensus.cc:482] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:52:05.039700 6545 raft_consensus.cc:504] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:05.041330 6545 leader_election.cc:288] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a566c5a4d4494120ac546c23153d5e04 (127.26.146.129:40873), 6378fdf302634a1f8dbf853985952b1a (127.26.146.131:45717)
I20260808 03:52:05.050268 6536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:05.041886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:49007 (local address 127.26.146.131:45717)
0808 03:52:05.042085 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:05.042093 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:05.043140 (+ 1047us) server_negotiation.cc:407] Connection header received
0808 03:52:05.043421 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:05.043427 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:05.043508 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:05.043600 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:52:05.044288 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.045395 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.048000 (+ 2605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.049191 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.049314 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:05.049377 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:05.049895 (+ 518us) server_negotiation.cc:299] Negotiation successful
0808 03:52:05.050017 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 03:52:05.050271 6535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:05.042442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:43815 (local address 127.26.146.129:40873)
0808 03:52:05.042634 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:05.042640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:05.043365 (+ 725us) server_negotiation.cc:407] Connection header received
0808 03:52:05.043604 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:05.043609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:05.043686 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:05.043768 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:52:05.044606 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.045604 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.047759 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.049125 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.049303 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:05.049377 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:05.049810 (+ 433us) server_negotiation.cc:299] Negotiation successful
0808 03:52:05.049963 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 03:52:05.050282 6600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:05.041722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:40873 (local address 127.26.146.130:43815)
0808 03:52:05.043273 (+ 1551us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:05.043289 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:05.043462 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:05.043947 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:05.043950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:05.043974 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:52:05.044380 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:05.044394 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.045759 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.045762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:05.047596 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:05.047618 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.049379 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.049385 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:05.049617 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:05.049653 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 03:52:05.049759 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 03:52:05.049895 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1441,"mutex_wait_us":87,"thread_start_us":1256,"threads_started":1}
I20260808 03:52:05.051362 6599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:05.041709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:45717 (local address 127.26.146.130:49007)
0808 03:52:05.043008 (+ 1299us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:05.043032 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:05.043263 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:05.043615 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:05.043619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:05.043639 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:52:05.044056 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:05.044067 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.045564 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.045568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:05.047839 (+ 2271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:05.047854 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.049531 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.049535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:05.049712 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:05.049742 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 03:52:05.050571 (+ 829us) client_negotiation.cc:240] Negotiation successful
0808 03:52:05.050709 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1133,"mutex_wait_us":58,"thread_start_us":929,"threads_started":1}
I20260808 03:52:05.051293 6040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c0980067d44475b9fb43b9a4ab467d3" candidate_uuid: "f10f8c8f4c674ec39f201d90d662bf42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a566c5a4d4494120ac546c23153d5e04" is_pre_election: true
I20260808 03:52:05.051591 6040 raft_consensus.cc:2454] T 6c0980067d44475b9fb43b9a4ab467d3 P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f10f8c8f4c674ec39f201d90d662bf42 in term 0.
I20260808 03:52:05.051993 6434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c0980067d44475b9fb43b9a4ab467d3" candidate_uuid: "f10f8c8f4c674ec39f201d90d662bf42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6378fdf302634a1f8dbf853985952b1a" is_pre_election: true
I20260808 03:52:05.052131 6137 leader_election.cc:302] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a566c5a4d4494120ac546c23153d5e04, f10f8c8f4c674ec39f201d90d662bf42; no voters:
I20260808 03:52:05.052265 6434 raft_consensus.cc:2454] T 6c0980067d44475b9fb43b9a4ab467d3 P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f10f8c8f4c674ec39f201d90d662bf42 in term 0.
I20260808 03:52:05.052600 6545 raft_consensus.cc:2781] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:52:05.052742 6545 raft_consensus.cc:482] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:52:05.052863 6545 raft_consensus.cc:3037] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:52:05.055888 6545 raft_consensus.cc:504] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:05.056612 6545 leader_election.cc:288] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42 [CANDIDATE]: Term 1 election: Requested vote from peers a566c5a4d4494120ac546c23153d5e04 (127.26.146.129:40873), 6378fdf302634a1f8dbf853985952b1a (127.26.146.131:45717)
I20260808 03:52:05.057075 6040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c0980067d44475b9fb43b9a4ab467d3" candidate_uuid: "f10f8c8f4c674ec39f201d90d662bf42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a566c5a4d4494120ac546c23153d5e04"
I20260808 03:52:05.057124 6434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c0980067d44475b9fb43b9a4ab467d3" candidate_uuid: "f10f8c8f4c674ec39f201d90d662bf42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6378fdf302634a1f8dbf853985952b1a"
I20260808 03:52:05.057308 6040 raft_consensus.cc:3037] T 6c0980067d44475b9fb43b9a4ab467d3 P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:52:05.057335 6434 raft_consensus.cc:3037] T 6c0980067d44475b9fb43b9a4ab467d3 P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:52:05.059728 6040 raft_consensus.cc:2454] T 6c0980067d44475b9fb43b9a4ab467d3 P a566c5a4d4494120ac546c23153d5e04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f10f8c8f4c674ec39f201d90d662bf42 in term 1.
I20260808 03:52:05.060143 6137 leader_election.cc:302] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a566c5a4d4494120ac546c23153d5e04, f10f8c8f4c674ec39f201d90d662bf42; no voters:
I20260808 03:52:05.060256 6434 raft_consensus.cc:2454] T 6c0980067d44475b9fb43b9a4ab467d3 P 6378fdf302634a1f8dbf853985952b1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f10f8c8f4c674ec39f201d90d662bf42 in term 1.
I20260808 03:52:05.060436 6545 raft_consensus.cc:2781] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:52:05.060930 6545 raft_consensus.cc:686] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42 [term 1 LEADER]: Becoming Leader. State: Replica: f10f8c8f4c674ec39f201d90d662bf42, State: Running, Role: LEADER
I20260808 03:52:05.061342 6545 consensus_queue.cc:237] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:05.063396 6601 tablet_replica.cc:439] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42: TxnStatusTablet state changed. Reason: New leader f10f8c8f4c674ec39f201d90d662bf42. Latest consensus state: current_term: 1 leader_uuid: "f10f8c8f4c674ec39f201d90d662bf42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } } }
I20260808 03:52:05.063969 6601 tablet_replica.cc:442] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42: This TxnStatusTablet replica's current role is: LEADER
I20260808 03:52:05.064738 6604 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 03:52:05.064970 6604 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 03:52:05.065258 5800 catalog_manager.cc:5495] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42 reported cstate change: term changed from 0 to 1, leader changed from <none> to f10f8c8f4c674ec39f201d90d662bf42 (127.26.146.130). New cstate: current_term: 1 leader_uuid: "f10f8c8f4c674ec39f201d90d662bf42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } health_report { overall_health: UNKNOWN } } }
I20260808 03:52:05.086191 6592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:05.080600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:35087 (local address 127.0.0.1:52114)
0808 03:52:05.080802 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:05.080822 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:05.080952 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:05.081471 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:05.081475 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:05.081488 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 03:52:05.081663 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:05.081672 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.082735 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.082741 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:05.083768 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:05.083780 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.084588 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.084594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:05.084762 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:05.084825 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 03:52:05.085644 (+ 819us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 03:52:05.085666 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 03:52:05.085743 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 03:52:05.085901 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 03:52:05.086192 6532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:05.080698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52114 (local address 127.26.146.130:35087)
0808 03:52:05.080857 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:05.080863 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:05.080881 (+ 18us) server_negotiation.cc:407] Connection header received
0808 03:52:05.081170 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:05.081176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:05.081303 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:05.081433 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:52:05.081803 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.082592 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.083945 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.084409 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.084537 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:05.085011 (+ 474us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:52:05.085528 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:52:05.085668 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:05.085766 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 03:52:05.085874 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 03:52:05.090036 6434 raft_consensus.cc:1261] T 6c0980067d44475b9fb43b9a4ab467d3 P 6378fdf302634a1f8dbf853985952b1a [term 1 FOLLOWER]: Refusing update from remote peer f10f8c8f4c674ec39f201d90d662bf42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:52:05.090525 6040 raft_consensus.cc:1261] T 6c0980067d44475b9fb43b9a4ab467d3 P a566c5a4d4494120ac546c23153d5e04 [term 1 FOLLOWER]: Refusing update from remote peer f10f8c8f4c674ec39f201d90d662bf42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:52:05.091042 6545 consensus_queue.cc:1035] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:52:05.091351 6601 consensus_queue.cc:1035] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:52:05.103376 6546 tablet_replica.cc:439] T 6c0980067d44475b9fb43b9a4ab467d3 P 6378fdf302634a1f8dbf853985952b1a: TxnStatusTablet state changed. Reason: New leader f10f8c8f4c674ec39f201d90d662bf42. Latest consensus state: current_term: 1 leader_uuid: "f10f8c8f4c674ec39f201d90d662bf42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } } }
I20260808 03:52:05.103734 6546 tablet_replica.cc:442] T 6c0980067d44475b9fb43b9a4ab467d3 P 6378fdf302634a1f8dbf853985952b1a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 03:52:05.105574 6550 tablet_replica.cc:439] T 6c0980067d44475b9fb43b9a4ab467d3 P a566c5a4d4494120ac546c23153d5e04: TxnStatusTablet state changed. Reason: New leader f10f8c8f4c674ec39f201d90d662bf42. Latest consensus state: current_term: 1 leader_uuid: "f10f8c8f4c674ec39f201d90d662bf42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } } }
I20260808 03:52:05.105865 6550 tablet_replica.cc:442] T 6c0980067d44475b9fb43b9a4ab467d3 P a566c5a4d4494120ac546c23153d5e04: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 03:52:05.105722 6545 tablet_replica.cc:439] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f10f8c8f4c674ec39f201d90d662bf42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } } }
I20260808 03:52:05.106029 6545 tablet_replica.cc:442] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42: This TxnStatusTablet replica's current role is: LEADER
I20260808 03:52:05.106410 6556 mvcc.cc:205] Tried to move back new op lower bound from 7316115968361816064 to 7316115968254988288. Current Snapshot: MvccSnapshot[applied={T|T < 7316115968361816064}]
I20260808 03:52:05.107774 6555 mvcc.cc:205] Tried to move back new op lower bound from 7316115968361816064 to 7316115968254988288. Current Snapshot: MvccSnapshot[applied={T|T < 7316115968361816064}]
I20260808 03:52:05.108070 6546 tablet_replica.cc:439] T 6c0980067d44475b9fb43b9a4ab467d3 P 6378fdf302634a1f8dbf853985952b1a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f10f8c8f4c674ec39f201d90d662bf42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } } }
I20260808 03:52:05.108330 6546 tablet_replica.cc:442] T 6c0980067d44475b9fb43b9a4ab467d3 P 6378fdf302634a1f8dbf853985952b1a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 03:52:05.113447 6545 tablet_replica.cc:439] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f10f8c8f4c674ec39f201d90d662bf42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } } }
I20260808 03:52:05.113754 6545 tablet_replica.cc:442] T 6c0980067d44475b9fb43b9a4ab467d3 P f10f8c8f4c674ec39f201d90d662bf42: This TxnStatusTablet replica's current role is: LEADER
I20260808 03:52:05.115048 6554 mvcc.cc:205] Tried to move back new op lower bound from 7316115968361816064 to 7316115968254988288. Current Snapshot: MvccSnapshot[applied={T|T < 7316115968361816064}]
I20260808 03:52:05.116418 6550 tablet_replica.cc:439] T 6c0980067d44475b9fb43b9a4ab467d3 P a566c5a4d4494120ac546c23153d5e04: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f10f8c8f4c674ec39f201d90d662bf42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } } }
I20260808 03:52:05.116719 6550 tablet_replica.cc:442] T 6c0980067d44475b9fb43b9a4ab467d3 P a566c5a4d4494120ac546c23153d5e04: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 03:52:05.126447 6536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:05.119203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39948 (local address 127.26.146.131:45717)
0808 03:52:05.119379 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:05.119385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:05.119403 (+ 18us) server_negotiation.cc:407] Connection header received
0808 03:52:05.119504 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:05.119509 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:05.119629 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:05.119796 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:52:05.120334 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.121425 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.123466 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.124215 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.124338 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:05.124998 (+ 660us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:52:05.125747 (+ 749us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:52:05.125903 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:05.126048 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 03:52:05.126137 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 03:52:05.147071 6612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:05.138257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:35087 (local address 127.26.146.131:54679)
0808 03:52:05.140304 (+ 2047us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:05.140324 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:05.140465 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:05.140840 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:05.140845 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:05.140857 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 03:52:05.141175 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:05.141186 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.142687 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.142693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:05.144412 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:05.144427 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.145604 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.145611 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:05.145797 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:05.145868 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 03:52:05.146356 (+ 488us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 03:52:05.146375 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 03:52:05.146458 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 03:52:05.146602 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1878,"thread_start_us":1737,"threads_started":1}
I20260808 03:52:05.148043 6532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:05.138504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:54679 (local address 127.26.146.130:35087)
0808 03:52:05.138708 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:05.138714 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:05.140423 (+ 1709us) server_negotiation.cc:407] Connection header received
0808 03:52:05.140604 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:05.140618 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:05.140711 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:05.140812 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:52:05.141351 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.142538 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.144605 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.145415 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.145543 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:05.146089 (+ 546us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:52:05.146251 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:52:05.146346 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:05.146511 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 03:52:05.146596 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 03:52:05.159621 6237 raft_consensus.cc:1261] T f95041d155fe404384ac334e07fc57ac P f10f8c8f4c674ec39f201d90d662bf42 [term 1 FOLLOWER]: Refusing update from remote peer 6378fdf302634a1f8dbf853985952b1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:52:05.159860 6039 raft_consensus.cc:1261] T f95041d155fe404384ac334e07fc57ac P a566c5a4d4494120ac546c23153d5e04 [term 1 FOLLOWER]: Refusing update from remote peer 6378fdf302634a1f8dbf853985952b1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:52:05.160450 6581 consensus_queue.cc:1035] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:52:05.161140 6581 consensus_queue.cc:1035] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:52:05.223358 6618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:05.214561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:45717 (local address 127.26.146.130:39431)
0808 03:52:05.216390 (+ 1829us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:05.216412 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:05.216621 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:05.217012 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:05.217017 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:05.217031 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 03:52:05.217411 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:05.217425 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.218845 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.218851 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:05.220748 (+ 1897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:05.220764 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.221878 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.221884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:05.222063 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:05.222144 (+ 81us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 03:52:05.222654 (+ 510us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 03:52:05.222666 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 03:52:05.222766 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 03:52:05.222918 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1617,"thread_start_us":1840,"threads_started":1}
I20260808 03:52:05.224197 6536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:05.213750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:39431 (local address 127.26.146.131:45717)
0808 03:52:05.213960 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:05.213967 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:05.216491 (+ 2524us) server_negotiation.cc:407] Connection header received
0808 03:52:05.216775 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:05.216781 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:05.216869 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:05.216972 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:52:05.217591 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.218713 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.220920 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.221682 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.221816 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:05.222414 (+ 598us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:52:05.222554 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:52:05.222646 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:05.222802 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 03:52:05.222887 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 03:52:05.293444 5800 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:52:05.293716 5800 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:52:05.297489 5800 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47: Sending DeleteTablet for 3 replicas of tablet f95041d155fe404384ac334e07fc57ac
I20260808 03:52:05.298879 6401 tablet_service.cc:1514] Processing DeleteTablet for tablet f95041d155fe404384ac334e07fc57ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:52:05 UTC) from {username='oryx', principal='oryx/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:39942
I20260808 03:52:05.299193 6569 tablet_replica.cc:330] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a: stopping tablet replica
I20260808 03:52:05.299393 6569 raft_consensus.cc:2229] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:52:05.300370 6003 tablet_service.cc:1514] Processing DeleteTablet for tablet f95041d155fe404384ac334e07fc57ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:52:05 UTC) from {username='oryx', principal='oryx/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:34554
I20260808 03:52:05.300683 6570 tablet_replica.cc:330] T f95041d155fe404384ac334e07fc57ac P a566c5a4d4494120ac546c23153d5e04: stopping tablet replica
I20260808 03:52:05.300779 6569 raft_consensus.cc:2258] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:52:05.300902 6570 raft_consensus.cc:2229] T f95041d155fe404384ac334e07fc57ac P a566c5a4d4494120ac546c23153d5e04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:52:05.301107 6570 raft_consensus.cc:2258] T f95041d155fe404384ac334e07fc57ac P a566c5a4d4494120ac546c23153d5e04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:52:05.302117 6569 ts_tablet_manager.cc:1905] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:52:05.302448 6570 ts_tablet_manager.cc:1905] T f95041d155fe404384ac334e07fc57ac P a566c5a4d4494120ac546c23153d5e04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:52:05.303376 6208 tablet_service.cc:1514] Processing DeleteTablet for tablet f95041d155fe404384ac334e07fc57ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:52:05 UTC) from {username='oryx', principal='oryx/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:52108
I20260808 03:52:05.303962 6572 tablet_replica.cc:330] T f95041d155fe404384ac334e07fc57ac P f10f8c8f4c674ec39f201d90d662bf42: stopping tablet replica
I20260808 03:52:05.304175 6572 raft_consensus.cc:2229] T f95041d155fe404384ac334e07fc57ac P f10f8c8f4c674ec39f201d90d662bf42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:52:05.304374 6572 raft_consensus.cc:2258] T f95041d155fe404384ac334e07fc57ac P f10f8c8f4c674ec39f201d90d662bf42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:52:05.307183 6572 ts_tablet_manager.cc:1905] T f95041d155fe404384ac334e07fc57ac P f10f8c8f4c674ec39f201d90d662bf42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:52:05.308622 6569 ts_tablet_manager.cc:1918] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 03:52:05.310040 6570 ts_tablet_manager.cc:1918] T f95041d155fe404384ac334e07fc57ac P a566c5a4d4494120ac546c23153d5e04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 03:52:05.310199 6570 log.cc:1196] T f95041d155fe404384ac334e07fc57ac P a566c5a4d4494120ac546c23153d5e04: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-0/wal/wals/f95041d155fe404384ac334e07fc57ac
I20260808 03:52:05.310607 6570 ts_tablet_manager.cc:1939] T f95041d155fe404384ac334e07fc57ac P a566c5a4d4494120ac546c23153d5e04: Deleting consensus metadata
I20260808 03:52:05.311733 5780 catalog_manager.cc:4841] TS a566c5a4d4494120ac546c23153d5e04 (127.26.146.129:40873): tablet f95041d155fe404384ac334e07fc57ac (table test-table [id=16f303951d964e71b13248493e2547a6]) successfully deleted
I20260808 03:52:05.314947 6572 ts_tablet_manager.cc:1918] T f95041d155fe404384ac334e07fc57ac P f10f8c8f4c674ec39f201d90d662bf42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 03:52:05.315109 6572 log.cc:1196] T f95041d155fe404384ac334e07fc57ac P f10f8c8f4c674ec39f201d90d662bf42: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/f95041d155fe404384ac334e07fc57ac
I20260808 03:52:05.315551 6572 ts_tablet_manager.cc:1939] T f95041d155fe404384ac334e07fc57ac P f10f8c8f4c674ec39f201d90d662bf42: Deleting consensus metadata
I20260808 03:52:05.316644 5783 catalog_manager.cc:4841] TS f10f8c8f4c674ec39f201d90d662bf42 (127.26.146.130:35087): tablet f95041d155fe404384ac334e07fc57ac (table test-table [id=16f303951d964e71b13248493e2547a6]) successfully deleted
I20260808 03:52:05.316915 6569 log.cc:1196] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/f95041d155fe404384ac334e07fc57ac
I20260808 03:52:05.317395 6569 ts_tablet_manager.cc:1939] T f95041d155fe404384ac334e07fc57ac P 6378fdf302634a1f8dbf853985952b1a: Deleting consensus metadata
I20260808 03:52:05.318512 5779 catalog_manager.cc:4841] TS 6378fdf302634a1f8dbf853985952b1a (127.26.146.131:45717): tablet f95041d155fe404384ac334e07fc57ac (table test-table [id=16f303951d964e71b13248493e2547a6]) successfully deleted
Aug 08 03:52:05 dist-test-slave-jlzn krb5kdc[5740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 03:52:05 dist-test-slave-jlzn krb5kdc[5740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:05 dist-test-slave-jlzn krb5kdc[5740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 03:52:05.386885 5939 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:05.373720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47680 (local address 127.26.146.190:42201)
0808 03:52:05.373957 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:05.373964 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:05.374673 (+ 709us) server_negotiation.cc:407] Connection header received
0808 03:52:05.374879 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:05.374885 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:05.374972 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:05.375082 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:05.376822 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.377972 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.379836 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.380530 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.380674 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:05.386566 (+ 5892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47680: BlockingRecv error: recv got EOF from 127.0.0.1:47680 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 08 03:52:05 dist-test-slave-jlzn krb5kdc[5740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM
I20260808 03:52:05.408906 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:05.390519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47684 (local address 127.26.146.190:42201)
0808 03:52:05.390711 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:05.390718 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:05.391275 (+ 557us) server_negotiation.cc:407] Connection header received
0808 03:52:05.391425 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:05.391430 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:05.391528 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:05.391638 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:05.393325 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.394412 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.396317 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.396954 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.397049 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:05.402054 (+ 5005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:05.402061 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:05.402066 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:05.402111 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:05.405656 (+ 3545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:05.406467 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:05.406472 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:05.406476 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:05.406626 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:05.407090 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:05.407095 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:05.407098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:05.407461 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:05.407694 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:05.408276 (+ 582us) server_negotiation.cc:299] Negotiation successful
0808 03:52:05.408558 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 03:52:05.414011 5800 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:52:05.414767 5800 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:52:05.423584 6208 tablet_service.cc:1467] Processing CreateTablet for tablet 9b0b48788771456f865ff86e03c84158 (DEFAULT_TABLE table=test-table [id=894caa7e0cea41d59ebb589be2d4540c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:52:05.424134 6208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b0b48788771456f865ff86e03c84158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:52:05.428251 6401 tablet_service.cc:1467] Processing CreateTablet for tablet 9b0b48788771456f865ff86e03c84158 (DEFAULT_TABLE table=test-table [id=894caa7e0cea41d59ebb589be2d4540c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:52:05.428723 6401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b0b48788771456f865ff86e03c84158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:52:05.432997 6003 tablet_service.cc:1467] Processing CreateTablet for tablet 9b0b48788771456f865ff86e03c84158 (DEFAULT_TABLE table=test-table [id=894caa7e0cea41d59ebb589be2d4540c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:52:05.433457 6003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b0b48788771456f865ff86e03c84158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:52:05.452728 6540 tablet_bootstrap.cc:492] T 9b0b48788771456f865ff86e03c84158 P a566c5a4d4494120ac546c23153d5e04: Bootstrap starting.
I20260808 03:52:05.454483 6540 tablet_bootstrap.cc:654] T 9b0b48788771456f865ff86e03c84158 P a566c5a4d4494120ac546c23153d5e04: Neither blocks nor log segments found. Creating new log.
I20260808 03:52:05.459213 6539 tablet_bootstrap.cc:492] T 9b0b48788771456f865ff86e03c84158 P 6378fdf302634a1f8dbf853985952b1a: Bootstrap starting.
I20260808 03:52:05.460969 6539 tablet_bootstrap.cc:654] T 9b0b48788771456f865ff86e03c84158 P 6378fdf302634a1f8dbf853985952b1a: Neither blocks nor log segments found. Creating new log.
I20260808 03:52:05.461169 6538 tablet_bootstrap.cc:492] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42: Bootstrap starting.
I20260808 03:52:05.462944 6538 tablet_bootstrap.cc:654] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42: Neither blocks nor log segments found. Creating new log.
I20260808 03:52:05.473104 6539 tablet_bootstrap.cc:492] T 9b0b48788771456f865ff86e03c84158 P 6378fdf302634a1f8dbf853985952b1a: No bootstrap required, opened a new log
I20260808 03:52:05.473284 6539 ts_tablet_manager.cc:1397] T 9b0b48788771456f865ff86e03c84158 P 6378fdf302634a1f8dbf853985952b1a: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260808 03:52:05.474114 6539 raft_consensus.cc:348] T 9b0b48788771456f865ff86e03c84158 P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:05.474385 6539 raft_consensus.cc:374] T 9b0b48788771456f865ff86e03c84158 P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:52:05.474493 6539 raft_consensus.cc:729] T 9b0b48788771456f865ff86e03c84158 P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6378fdf302634a1f8dbf853985952b1a, State: Initialized, Role: FOLLOWER
I20260808 03:52:05.474773 6539 consensus_queue.cc:260] T 9b0b48788771456f865ff86e03c84158 P 6378fdf302634a1f8dbf853985952b1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:05.475783 6539 ts_tablet_manager.cc:1428] T 9b0b48788771456f865ff86e03c84158 P 6378fdf302634a1f8dbf853985952b1a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 03:52:05.476044 6540 tablet_bootstrap.cc:492] T 9b0b48788771456f865ff86e03c84158 P a566c5a4d4494120ac546c23153d5e04: No bootstrap required, opened a new log
I20260808 03:52:05.476263 6540 ts_tablet_manager.cc:1397] T 9b0b48788771456f865ff86e03c84158 P a566c5a4d4494120ac546c23153d5e04: Time spent bootstrapping tablet: real 0.024s user 0.008s sys 0.001s
I20260808 03:52:05.477152 6540 raft_consensus.cc:348] T 9b0b48788771456f865ff86e03c84158 P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:05.477465 6540 raft_consensus.cc:374] T 9b0b48788771456f865ff86e03c84158 P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:52:05.477942 6540 raft_consensus.cc:729] T 9b0b48788771456f865ff86e03c84158 P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a566c5a4d4494120ac546c23153d5e04, State: Initialized, Role: FOLLOWER
I20260808 03:52:05.478293 6540 consensus_queue.cc:260] T 9b0b48788771456f865ff86e03c84158 P a566c5a4d4494120ac546c23153d5e04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:05.478487 6538 tablet_bootstrap.cc:492] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42: No bootstrap required, opened a new log
I20260808 03:52:05.478683 6538 ts_tablet_manager.cc:1397] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.003s
I20260808 03:52:05.479110 6540 ts_tablet_manager.cc:1428] T 9b0b48788771456f865ff86e03c84158 P a566c5a4d4494120ac546c23153d5e04: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260808 03:52:05.479480 6538 raft_consensus.cc:348] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:05.479796 6538 raft_consensus.cc:374] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:52:05.479938 6538 raft_consensus.cc:729] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f10f8c8f4c674ec39f201d90d662bf42, State: Initialized, Role: FOLLOWER
I20260808 03:52:05.480238 6538 consensus_queue.cc:260] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:05.481720 6538 ts_tablet_manager.cc:1428] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 03:52:05.512838 6545 raft_consensus.cc:482] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:52:05.513079 6545 raft_consensus.cc:504] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:05.513865 6545 leader_election.cc:288] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a566c5a4d4494120ac546c23153d5e04 (127.26.146.129:40873), 6378fdf302634a1f8dbf853985952b1a (127.26.146.131:45717)
I20260808 03:52:05.514349 6040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b0b48788771456f865ff86e03c84158" candidate_uuid: "f10f8c8f4c674ec39f201d90d662bf42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a566c5a4d4494120ac546c23153d5e04" is_pre_election: true
I20260808 03:52:05.514632 6040 raft_consensus.cc:2454] T 9b0b48788771456f865ff86e03c84158 P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f10f8c8f4c674ec39f201d90d662bf42 in term 0.
I20260808 03:52:05.514575 6434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b0b48788771456f865ff86e03c84158" candidate_uuid: "f10f8c8f4c674ec39f201d90d662bf42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6378fdf302634a1f8dbf853985952b1a" is_pre_election: true
I20260808 03:52:05.514818 6434 raft_consensus.cc:2454] T 9b0b48788771456f865ff86e03c84158 P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f10f8c8f4c674ec39f201d90d662bf42 in term 0.
I20260808 03:52:05.515336 6136 leader_election.cc:302] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6378fdf302634a1f8dbf853985952b1a, a566c5a4d4494120ac546c23153d5e04, f10f8c8f4c674ec39f201d90d662bf42; no voters:
I20260808 03:52:05.515843 6545 raft_consensus.cc:2781] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:52:05.515990 6545 raft_consensus.cc:482] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:52:05.516112 6545 raft_consensus.cc:3037] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:52:05.519338 6545 raft_consensus.cc:504] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:05.520483 6040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b0b48788771456f865ff86e03c84158" candidate_uuid: "f10f8c8f4c674ec39f201d90d662bf42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a566c5a4d4494120ac546c23153d5e04"
I20260808 03:52:05.520588 6434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b0b48788771456f865ff86e03c84158" candidate_uuid: "f10f8c8f4c674ec39f201d90d662bf42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6378fdf302634a1f8dbf853985952b1a"
I20260808 03:52:05.520743 6040 raft_consensus.cc:3037] T 9b0b48788771456f865ff86e03c84158 P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:52:05.520828 6434 raft_consensus.cc:3037] T 9b0b48788771456f865ff86e03c84158 P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:52:05.523622 6040 raft_consensus.cc:2454] T 9b0b48788771456f865ff86e03c84158 P a566c5a4d4494120ac546c23153d5e04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f10f8c8f4c674ec39f201d90d662bf42 in term 1.
I20260808 03:52:05.524151 6434 raft_consensus.cc:2454] T 9b0b48788771456f865ff86e03c84158 P 6378fdf302634a1f8dbf853985952b1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f10f8c8f4c674ec39f201d90d662bf42 in term 1.
I20260808 03:52:05.524224 6137 leader_election.cc:302] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a566c5a4d4494120ac546c23153d5e04, f10f8c8f4c674ec39f201d90d662bf42; no voters:
I20260808 03:52:05.524529 6601 raft_consensus.cc:2781] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:52:05.524695 6545 leader_election.cc:288] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42 [CANDIDATE]: Term 1 election: Requested vote from peers a566c5a4d4494120ac546c23153d5e04 (127.26.146.129:40873), 6378fdf302634a1f8dbf853985952b1a (127.26.146.131:45717)
I20260808 03:52:05.528106 6601 raft_consensus.cc:686] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42 [term 1 LEADER]: Becoming Leader. State: Replica: f10f8c8f4c674ec39f201d90d662bf42, State: Running, Role: LEADER
I20260808 03:52:05.528445 6601 consensus_queue.cc:237] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:05.531318 5800 catalog_manager.cc:5495] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42 reported cstate change: term changed from 0 to 1, leader changed from <none> to f10f8c8f4c674ec39f201d90d662bf42 (127.26.146.130). New cstate: current_term: 1 leader_uuid: "f10f8c8f4c674ec39f201d90d662bf42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } health_report { overall_health: UNKNOWN } } }
I20260808 03:52:05.558787 6532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:05.552557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52118 (local address 127.26.146.130:35087)
0808 03:52:05.552734 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:05.552741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:05.552822 (+ 81us) server_negotiation.cc:407] Connection header received
0808 03:52:05.553057 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:05.553063 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:05.553147 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:05.553240 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:52:05.553659 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.554696 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.556660 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.557322 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.557454 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:05.557976 (+ 522us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:52:05.558125 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:52:05.558220 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:05.558398 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 03:52:05.558483 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 03:52:05.562450 6434 raft_consensus.cc:1261] T 9b0b48788771456f865ff86e03c84158 P 6378fdf302634a1f8dbf853985952b1a [term 1 FOLLOWER]: Refusing update from remote peer f10f8c8f4c674ec39f201d90d662bf42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:52:05.562561 6040 raft_consensus.cc:1261] T 9b0b48788771456f865ff86e03c84158 P a566c5a4d4494120ac546c23153d5e04 [term 1 FOLLOWER]: Refusing update from remote peer f10f8c8f4c674ec39f201d90d662bf42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:52:05.563223 6644 consensus_queue.cc:1035] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:52:05.563975 6545 consensus_queue.cc:1035] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:52:05.596285 5799 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:52:05.596537 5799 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:52:05.599964 5799 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47: Sending DeleteTablet for 3 replicas of tablet 9b0b48788771456f865ff86e03c84158
I20260808 03:52:05.600870 6003 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b0b48788771456f865ff86e03c84158 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:52:05 UTC) from {username='oryx', principal='oryx/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:34554
I20260808 03:52:05.601208 6208 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b0b48788771456f865ff86e03c84158 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:52:05 UTC) from {username='oryx', principal='oryx/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:52108
I20260808 03:52:05.601526 6570 tablet_replica.cc:330] T 9b0b48788771456f865ff86e03c84158 P a566c5a4d4494120ac546c23153d5e04: stopping tablet replica
I20260808 03:52:05.601722 6570 raft_consensus.cc:2229] T 9b0b48788771456f865ff86e03c84158 P a566c5a4d4494120ac546c23153d5e04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:52:05.601976 6570 raft_consensus.cc:2258] T 9b0b48788771456f865ff86e03c84158 P a566c5a4d4494120ac546c23153d5e04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:52:05.602300 6572 tablet_replica.cc:330] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42: stopping tablet replica
I20260808 03:52:05.602487 6572 raft_consensus.cc:2229] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42 [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:52:05.602984 6572 raft_consensus.cc:2258] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:52:05.603968 6401 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b0b48788771456f865ff86e03c84158 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:52:05 UTC) from {username='oryx', principal='oryx/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:39942
I20260808 03:52:05.604276 6569 tablet_replica.cc:330] T 9b0b48788771456f865ff86e03c84158 P 6378fdf302634a1f8dbf853985952b1a: stopping tablet replica
I20260808 03:52:05.604465 6569 raft_consensus.cc:2229] T 9b0b48788771456f865ff86e03c84158 P 6378fdf302634a1f8dbf853985952b1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:52:05.604703 6569 raft_consensus.cc:2258] T 9b0b48788771456f865ff86e03c84158 P 6378fdf302634a1f8dbf853985952b1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:52:05.606105 5799 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 03:52:05.606369 6572 ts_tablet_manager.cc:1905] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 03:52:05.606920 5799 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:52:05.609634 6570 ts_tablet_manager.cc:1905] T 9b0b48788771456f865ff86e03c84158 P a566c5a4d4494120ac546c23153d5e04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:52:05.613540 6572 ts_tablet_manager.cc:1918] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:52:05.613685 6572 log.cc:1196] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/9b0b48788771456f865ff86e03c84158
I20260808 03:52:05.614107 6572 ts_tablet_manager.cc:1939] T 9b0b48788771456f865ff86e03c84158 P f10f8c8f4c674ec39f201d90d662bf42: Deleting consensus metadata
I20260808 03:52:05.614590 6208 tablet_service.cc:1467] Processing CreateTablet for tablet 8ad0407c2dd844c58825739db79b4a74 (DEFAULT_TABLE table=test-table [id=25c8e3a164344cbb93395829935c63d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:52:05.614988 6208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ad0407c2dd844c58825739db79b4a74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:52:05.615770 6570 ts_tablet_manager.cc:1918] T 9b0b48788771456f865ff86e03c84158 P a566c5a4d4494120ac546c23153d5e04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:52:05.615922 6570 log.cc:1196] T 9b0b48788771456f865ff86e03c84158 P a566c5a4d4494120ac546c23153d5e04: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-0/wal/wals/9b0b48788771456f865ff86e03c84158
I20260808 03:52:05.616357 6570 ts_tablet_manager.cc:1939] T 9b0b48788771456f865ff86e03c84158 P a566c5a4d4494120ac546c23153d5e04: Deleting consensus metadata
I20260808 03:52:05.618109 6569 ts_tablet_manager.cc:1905] T 9b0b48788771456f865ff86e03c84158 P 6378fdf302634a1f8dbf853985952b1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:52:05.620620 6401 tablet_service.cc:1467] Processing CreateTablet for tablet 8ad0407c2dd844c58825739db79b4a74 (DEFAULT_TABLE table=test-table [id=25c8e3a164344cbb93395829935c63d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:52:05.621070 6401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ad0407c2dd844c58825739db79b4a74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:52:05.624135 5783 catalog_manager.cc:4841] TS f10f8c8f4c674ec39f201d90d662bf42 (127.26.146.130:35087): tablet 9b0b48788771456f865ff86e03c84158 (table test-table [id=894caa7e0cea41d59ebb589be2d4540c]) successfully deleted
I20260808 03:52:05.624825 5780 catalog_manager.cc:4841] TS a566c5a4d4494120ac546c23153d5e04 (127.26.146.129:40873): tablet 9b0b48788771456f865ff86e03c84158 (table test-table [id=894caa7e0cea41d59ebb589be2d4540c]) successfully deleted
I20260808 03:52:05.625367 6569 ts_tablet_manager.cc:1918] T 9b0b48788771456f865ff86e03c84158 P 6378fdf302634a1f8dbf853985952b1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:52:05.625497 6569 log.cc:1196] T 9b0b48788771456f865ff86e03c84158 P 6378fdf302634a1f8dbf853985952b1a: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/9b0b48788771456f865ff86e03c84158
I20260808 03:52:05.625528 6003 tablet_service.cc:1467] Processing CreateTablet for tablet 8ad0407c2dd844c58825739db79b4a74 (DEFAULT_TABLE table=test-table [id=25c8e3a164344cbb93395829935c63d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:52:05.625964 6003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ad0407c2dd844c58825739db79b4a74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:52:05.625969 6569 ts_tablet_manager.cc:1939] T 9b0b48788771456f865ff86e03c84158 P 6378fdf302634a1f8dbf853985952b1a: Deleting consensus metadata
I20260808 03:52:05.627419 5779 catalog_manager.cc:4841] TS 6378fdf302634a1f8dbf853985952b1a (127.26.146.131:45717): tablet 9b0b48788771456f865ff86e03c84158 (table test-table [id=894caa7e0cea41d59ebb589be2d4540c]) successfully deleted
I20260808 03:52:05.632136 6540 tablet_bootstrap.cc:492] T 8ad0407c2dd844c58825739db79b4a74 P a566c5a4d4494120ac546c23153d5e04: Bootstrap starting.
I20260808 03:52:05.633761 6540 tablet_bootstrap.cc:654] T 8ad0407c2dd844c58825739db79b4a74 P a566c5a4d4494120ac546c23153d5e04: Neither blocks nor log segments found. Creating new log.
I20260808 03:52:05.634676 6539 tablet_bootstrap.cc:492] T 8ad0407c2dd844c58825739db79b4a74 P 6378fdf302634a1f8dbf853985952b1a: Bootstrap starting.
I20260808 03:52:05.634816 6538 tablet_bootstrap.cc:492] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42: Bootstrap starting.
I20260808 03:52:05.636423 6539 tablet_bootstrap.cc:654] T 8ad0407c2dd844c58825739db79b4a74 P 6378fdf302634a1f8dbf853985952b1a: Neither blocks nor log segments found. Creating new log.
I20260808 03:52:05.636937 6538 tablet_bootstrap.cc:654] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42: Neither blocks nor log segments found. Creating new log.
I20260808 03:52:05.652521 6539 tablet_bootstrap.cc:492] T 8ad0407c2dd844c58825739db79b4a74 P 6378fdf302634a1f8dbf853985952b1a: No bootstrap required, opened a new log
I20260808 03:52:05.652818 6539 ts_tablet_manager.cc:1397] T 8ad0407c2dd844c58825739db79b4a74 P 6378fdf302634a1f8dbf853985952b1a: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.003s
I20260808 03:52:05.653770 6539 raft_consensus.cc:348] T 8ad0407c2dd844c58825739db79b4a74 P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:05.654160 6539 raft_consensus.cc:374] T 8ad0407c2dd844c58825739db79b4a74 P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:52:05.654306 6539 raft_consensus.cc:729] T 8ad0407c2dd844c58825739db79b4a74 P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6378fdf302634a1f8dbf853985952b1a, State: Initialized, Role: FOLLOWER
I20260808 03:52:05.655727 6539 consensus_queue.cc:260] T 8ad0407c2dd844c58825739db79b4a74 P 6378fdf302634a1f8dbf853985952b1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:05.658927 6539 ts_tablet_manager.cc:1428] T 8ad0407c2dd844c58825739db79b4a74 P 6378fdf302634a1f8dbf853985952b1a: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260808 03:52:05.659894 6538 tablet_bootstrap.cc:492] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42: No bootstrap required, opened a new log
I20260808 03:52:05.660095 6538 ts_tablet_manager.cc:1397] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.012s
I20260808 03:52:05.661801 6538 raft_consensus.cc:348] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:05.662073 6538 raft_consensus.cc:374] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:52:05.662175 6538 raft_consensus.cc:729] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f10f8c8f4c674ec39f201d90d662bf42, State: Initialized, Role: FOLLOWER
I20260808 03:52:05.662487 6538 consensus_queue.cc:260] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:05.663268 6540 tablet_bootstrap.cc:492] T 8ad0407c2dd844c58825739db79b4a74 P a566c5a4d4494120ac546c23153d5e04: No bootstrap required, opened a new log
I20260808 03:52:05.663492 6540 ts_tablet_manager.cc:1397] T 8ad0407c2dd844c58825739db79b4a74 P a566c5a4d4494120ac546c23153d5e04: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.011s
I20260808 03:52:05.664435 6540 raft_consensus.cc:348] T 8ad0407c2dd844c58825739db79b4a74 P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:05.664750 6540 raft_consensus.cc:374] T 8ad0407c2dd844c58825739db79b4a74 P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:52:05.664884 6540 raft_consensus.cc:729] T 8ad0407c2dd844c58825739db79b4a74 P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a566c5a4d4494120ac546c23153d5e04, State: Initialized, Role: FOLLOWER
I20260808 03:52:05.665216 6540 consensus_queue.cc:260] T 8ad0407c2dd844c58825739db79b4a74 P a566c5a4d4494120ac546c23153d5e04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:05.665449 6538 ts_tablet_manager.cc:1428] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260808 03:52:05.666548 6540 ts_tablet_manager.cc:1428] T 8ad0407c2dd844c58825739db79b4a74 P a566c5a4d4494120ac546c23153d5e04: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 03:52:05.672773 6601 raft_consensus.cc:482] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:52:05.672964 6601 raft_consensus.cc:504] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:05.673640 6601 leader_election.cc:288] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a566c5a4d4494120ac546c23153d5e04 (127.26.146.129:40873), 6378fdf302634a1f8dbf853985952b1a (127.26.146.131:45717)
I20260808 03:52:05.674463 6039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ad0407c2dd844c58825739db79b4a74" candidate_uuid: "f10f8c8f4c674ec39f201d90d662bf42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a566c5a4d4494120ac546c23153d5e04" is_pre_election: true
I20260808 03:52:05.674518 6434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ad0407c2dd844c58825739db79b4a74" candidate_uuid: "f10f8c8f4c674ec39f201d90d662bf42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6378fdf302634a1f8dbf853985952b1a" is_pre_election: true
I20260808 03:52:05.674705 6039 raft_consensus.cc:2454] T 8ad0407c2dd844c58825739db79b4a74 P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f10f8c8f4c674ec39f201d90d662bf42 in term 0.
I20260808 03:52:05.674798 6434 raft_consensus.cc:2454] T 8ad0407c2dd844c58825739db79b4a74 P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f10f8c8f4c674ec39f201d90d662bf42 in term 0.
I20260808 03:52:05.675347 6136 leader_election.cc:302] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6378fdf302634a1f8dbf853985952b1a, f10f8c8f4c674ec39f201d90d662bf42; no voters:
I20260808 03:52:05.675693 6601 raft_consensus.cc:2781] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:52:05.675840 6601 raft_consensus.cc:482] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:52:05.675935 6601 raft_consensus.cc:3037] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:52:05.679013 6601 raft_consensus.cc:504] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:05.679625 6601 leader_election.cc:288] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42 [CANDIDATE]: Term 1 election: Requested vote from peers a566c5a4d4494120ac546c23153d5e04 (127.26.146.129:40873), 6378fdf302634a1f8dbf853985952b1a (127.26.146.131:45717)
I20260808 03:52:05.680171 6039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ad0407c2dd844c58825739db79b4a74" candidate_uuid: "f10f8c8f4c674ec39f201d90d662bf42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a566c5a4d4494120ac546c23153d5e04"
I20260808 03:52:05.680322 6434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ad0407c2dd844c58825739db79b4a74" candidate_uuid: "f10f8c8f4c674ec39f201d90d662bf42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6378fdf302634a1f8dbf853985952b1a"
I20260808 03:52:05.680392 6039 raft_consensus.cc:3037] T 8ad0407c2dd844c58825739db79b4a74 P a566c5a4d4494120ac546c23153d5e04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:52:05.680521 6434 raft_consensus.cc:3037] T 8ad0407c2dd844c58825739db79b4a74 P 6378fdf302634a1f8dbf853985952b1a [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:52:05.683350 6039 raft_consensus.cc:2454] T 8ad0407c2dd844c58825739db79b4a74 P a566c5a4d4494120ac546c23153d5e04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f10f8c8f4c674ec39f201d90d662bf42 in term 1.
I20260808 03:52:05.683916 6137 leader_election.cc:302] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a566c5a4d4494120ac546c23153d5e04, f10f8c8f4c674ec39f201d90d662bf42; no voters:
I20260808 03:52:05.684353 6601 raft_consensus.cc:2781] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:52:05.684821 6434 raft_consensus.cc:2454] T 8ad0407c2dd844c58825739db79b4a74 P 6378fdf302634a1f8dbf853985952b1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f10f8c8f4c674ec39f201d90d662bf42 in term 1.
I20260808 03:52:05.684906 6601 raft_consensus.cc:686] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42 [term 1 LEADER]: Becoming Leader. State: Replica: f10f8c8f4c674ec39f201d90d662bf42, State: Running, Role: LEADER
I20260808 03:52:05.685319 6601 consensus_queue.cc:237] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } }
I20260808 03:52:05.688252 5803 catalog_manager.cc:5495] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42 reported cstate change: term changed from 0 to 1, leader changed from <none> to f10f8c8f4c674ec39f201d90d662bf42 (127.26.146.130). New cstate: current_term: 1 leader_uuid: "f10f8c8f4c674ec39f201d90d662bf42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f10f8c8f4c674ec39f201d90d662bf42" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 } health_report { overall_health: UNKNOWN } } }
I20260808 03:52:05.724613 6618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:05.718057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:35087 (local address 127.26.146.130:53311)
0808 03:52:05.718236 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:05.718256 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:05.718344 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:05.718654 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:05.718659 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:05.718672 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 03:52:05.718894 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:05.718908 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.720505 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.720511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:05.722158 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:05.722173 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.723237 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.723244 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:05.723434 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:05.723487 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 03:52:05.724039 (+ 552us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 03:52:05.724047 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 03:52:05.724110 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 03:52:05.724257 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":1}
I20260808 03:52:05.725183 6532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:05.718124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:53311 (local address 127.26.146.130:35087)
0808 03:52:05.718294 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:05.718300 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:05.718319 (+ 19us) server_negotiation.cc:407] Connection header received
0808 03:52:05.718435 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:05.718440 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:05.718515 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:05.718618 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:52:05.719052 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.720346 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.722354 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.723058 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.723188 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:05.723681 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:52:05.723915 (+ 234us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:52:05.724002 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:05.724837 (+ 835us) server_negotiation.cc:299] Negotiation successful
0808 03:52:05.724910 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 03:52:05.737310 6434 raft_consensus.cc:1261] T 8ad0407c2dd844c58825739db79b4a74 P 6378fdf302634a1f8dbf853985952b1a [term 1 FOLLOWER]: Refusing update from remote peer f10f8c8f4c674ec39f201d90d662bf42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:52:05.737911 6039 raft_consensus.cc:1261] T 8ad0407c2dd844c58825739db79b4a74 P a566c5a4d4494120ac546c23153d5e04 [term 1 FOLLOWER]: Refusing update from remote peer f10f8c8f4c674ec39f201d90d662bf42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:52:05.738509 6601 consensus_queue.cc:1035] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a566c5a4d4494120ac546c23153d5e04" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:52:05.739626 6601 consensus_queue.cc:1035] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6378fdf302634a1f8dbf853985952b1a" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:52:05.827582 5803 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:52:05.827893 5803 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:52:05.831383 5803 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9c7ea7ea0cf54c879112453ed0d0ed47: Sending DeleteTablet for 3 replicas of tablet 8ad0407c2dd844c58825739db79b4a74
I20260808 03:52:05.832221 6003 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ad0407c2dd844c58825739db79b4a74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:52:05 UTC) from {username='oryx', principal='oryx/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:34554
I20260808 03:52:05.832536 6570 tablet_replica.cc:330] T 8ad0407c2dd844c58825739db79b4a74 P a566c5a4d4494120ac546c23153d5e04: stopping tablet replica
I20260808 03:52:05.832538 6401 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ad0407c2dd844c58825739db79b4a74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:52:05 UTC) from {username='oryx', principal='oryx/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:39942
I20260808 03:52:05.832749 6570 raft_consensus.cc:2229] T 8ad0407c2dd844c58825739db79b4a74 P a566c5a4d4494120ac546c23153d5e04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:52:05.832851 6569 tablet_replica.cc:330] T 8ad0407c2dd844c58825739db79b4a74 P 6378fdf302634a1f8dbf853985952b1a: stopping tablet replica
I20260808 03:52:05.832943 6570 raft_consensus.cc:2258] T 8ad0407c2dd844c58825739db79b4a74 P a566c5a4d4494120ac546c23153d5e04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:52:05.833056 6569 raft_consensus.cc:2229] T 8ad0407c2dd844c58825739db79b4a74 P 6378fdf302634a1f8dbf853985952b1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:52:05.833257 6569 raft_consensus.cc:2258] T 8ad0407c2dd844c58825739db79b4a74 P 6378fdf302634a1f8dbf853985952b1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:52:05.834213 6208 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ad0407c2dd844c58825739db79b4a74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:52:05 UTC) from {username='oryx', principal='oryx/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:52108
I20260808 03:52:05.834501 6572 tablet_replica.cc:330] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42: stopping tablet replica
I20260808 03:52:05.834699 6572 raft_consensus.cc:2229] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42 [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:52:05.835050 6572 raft_consensus.cc:2258] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:52:05.835588 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 5906
I20260808 03:52:05.836288 6572 ts_tablet_manager.cc:1905] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:52:05.841099 6570 ts_tablet_manager.cc:1905] T 8ad0407c2dd844c58825739db79b4a74 P a566c5a4d4494120ac546c23153d5e04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:52:05.851387 6572 ts_tablet_manager.cc:1918] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 03:52:05.851586 6572 log.cc:1196] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/8ad0407c2dd844c58825739db79b4a74
I20260808 03:52:05.852054 6572 ts_tablet_manager.cc:1939] T 8ad0407c2dd844c58825739db79b4a74 P f10f8c8f4c674ec39f201d90d662bf42: Deleting consensus metadata
I20260808 03:52:05.853268 5783 catalog_manager.cc:4841] TS f10f8c8f4c674ec39f201d90d662bf42 (127.26.146.130:35087): tablet 8ad0407c2dd844c58825739db79b4a74 (table test-table [id=25c8e3a164344cbb93395829935c63d6]) successfully deleted
I20260808 03:52:05.854781 6570 ts_tablet_manager.cc:1918] T 8ad0407c2dd844c58825739db79b4a74 P a566c5a4d4494120ac546c23153d5e04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 03:52:05.854949 6570 log.cc:1196] T 8ad0407c2dd844c58825739db79b4a74 P a566c5a4d4494120ac546c23153d5e04: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-0/wal/wals/8ad0407c2dd844c58825739db79b4a74
I20260808 03:52:05.855342 6570 ts_tablet_manager.cc:1939] T 8ad0407c2dd844c58825739db79b4a74 P a566c5a4d4494120ac546c23153d5e04: Deleting consensus metadata
I20260808 03:52:05.856474 5780 catalog_manager.cc:4841] TS a566c5a4d4494120ac546c23153d5e04 (127.26.146.129:40873): tablet 8ad0407c2dd844c58825739db79b4a74 (table test-table [id=25c8e3a164344cbb93395829935c63d6]) successfully deleted
I20260808 03:52:05.857515 6569 ts_tablet_manager.cc:1905] T 8ad0407c2dd844c58825739db79b4a74 P 6378fdf302634a1f8dbf853985952b1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:52:05.865378 6569 ts_tablet_manager.cc:1918] T 8ad0407c2dd844c58825739db79b4a74 P 6378fdf302634a1f8dbf853985952b1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 03:52:05.865557 6569 log.cc:1196] T 8ad0407c2dd844c58825739db79b4a74 P 6378fdf302634a1f8dbf853985952b1a: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/8ad0407c2dd844c58825739db79b4a74
I20260808 03:52:05.865991 6569 ts_tablet_manager.cc:1939] T 8ad0407c2dd844c58825739db79b4a74 P 6378fdf302634a1f8dbf853985952b1a: Deleting consensus metadata
I20260808 03:52:05.867427 5779 catalog_manager.cc:4841] TS 6378fdf302634a1f8dbf853985952b1a (127.26.146.131:45717): tablet 8ad0407c2dd844c58825739db79b4a74 (table test-table [id=25c8e3a164344cbb93395829935c63d6]) successfully deleted
Aug 08 03:52:05 dist-test-slave-jlzn krb5kdc[5740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.146.129@KRBTEST.COM
I20260808 03:52:05.881177 6666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:05.837923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34568 (local address 127.26.146.129:40873)
0808 03:52:05.861001 (+ 23078us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:05.861008 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:05.861804 (+ 796us) server_negotiation.cc:407] Connection header received
0808 03:52:05.861878 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:05.861884 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:05.861965 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:05.862065 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:05.864410 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.865713 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.867533 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:05.868241 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:05.868373 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:05.873717 (+ 5344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:05.873738 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:05.873758 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:05.873805 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:05.877830 (+ 4025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:05.878698 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:05.878709 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:05.878713 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:05.878892 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:05.879339 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:05.879344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:05.879347 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:05.879876 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:05.880185 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:05.880523 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 03:52:05.880773 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22947,"thread_start_us":23092,"threads_started":1}
I20260808 03:52:05.881970 6094 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:06.011878 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 5906
I20260808 03:52:06.040712 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 6106
Aug 08 03:52:06 dist-test-slave-jlzn krb5kdc[5740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.146.130@KRBTEST.COM
I20260808 03:52:06.059229 6532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:06.041659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52132 (local address 127.26.146.130:35087)
0808 03:52:06.041851 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:06.041858 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:06.041875 (+ 17us) server_negotiation.cc:407] Connection header received
0808 03:52:06.041938 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:06.041942 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:06.042003 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:06.042099 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:06.043773 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:06.044658 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:06.046551 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:06.047256 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:06.047389 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:06.052933 (+ 5544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:06.052948 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:06.052966 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:06.053004 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:06.056214 (+ 3210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:06.057033 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:06.057042 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:06.057044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:06.057199 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:06.057652 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:06.057656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:06.057658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:06.058071 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:06.058379 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:06.058676 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 03:52:06.058880 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 03:52:06.060137 6294 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:06.218370 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 6106
I20260808 03:52:06.249248 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 6309
Aug 08 03:52:06 dist-test-slave-jlzn krb5kdc[5740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.146.131@KRBTEST.COM
I20260808 03:52:06.264001 6670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:06.249958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39952 (local address 127.26.146.131:45717)
0808 03:52:06.250448 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:06.250454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:06.250470 (+ 16us) server_negotiation.cc:407] Connection header received
0808 03:52:06.250527 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:06.250531 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:06.250592 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:06.250706 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:06.252095 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:06.252884 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:06.254089 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:06.254515 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:06.254617 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:06.258780 (+ 4163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:06.258797 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:06.258814 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:06.258869 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:06.261528 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:06.262119 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:06.262129 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:06.262132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:06.262274 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:06.262566 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:06.262569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:06.262571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:06.262965 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:06.263203 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:06.263441 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 03:52:06.263637 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":242,"threads_started":1}
I20260808 03:52:06.264639 6493 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:06.386983 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 6309
I20260808 03:52:06.415660 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 5757
I20260808 03:52:06.416860 5863 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:06.535068 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 5757
2026-08-08T03:52:06Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (5703 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:52:06 dist-test-slave-jlzn krb5kdc[6678](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:52:06 dist-test-slave-jlzn krb5kdc[6678](info): set up 2 sockets
Aug 08 03:52:06 dist-test-slave-jlzn krb5kdc[6678](info): commencing operation
krb5kdc: starting...
W20260808 03:52:07.737457 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.001s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:52:07 dist-test-slave-jlzn krb5kdc[6678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:52:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:52:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:52:07.986831 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:39775
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:36135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:39775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:52:08.311053 6698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:08.311354 6698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:08.311512 6698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:08.318702 6698 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:52:08.318817 6698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:08.318898 6698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:08.318977 6698 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:52:08.319043 6698 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:52:08.328848 6698 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:36135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:39775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:39775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:08.330128 6698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:08.331422 6698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:08.342723 6706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:08.342774 6709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:08.342798 6707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:08.343585 6698 server_base.cc:1034] running on GCE node
I20260808 03:52:08.344365 6698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:08.345674 6698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:08.346876 6698 hybrid_clock.cc:648] HybridClock initialized: now 1786161128346836 us; error 30 us; skew 500 ppm
I20260808 03:52:08.347177 6698 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T03:52:08Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1873 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:52:08 dist-test-slave-jlzn krb5kdc[6715](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:52:08 dist-test-slave-jlzn krb5kdc[6715](info): set up 2 sockets
Aug 08 03:52:08 dist-test-slave-jlzn krb5kdc[6715](info): commencing operation
krb5kdc: starting...
W20260808 03:52:09.602931 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:52:09 dist-test-slave-jlzn krb5kdc[6715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:52:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:52:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:52:09.855182 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:36405
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:36405
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:52:10.179023 6732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:10.179312 6732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:10.179407 6732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:10.186440 6732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:52:10.186549 6732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:10.186594 6732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:10.186651 6732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:52:10.186694 6732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:52:10.196797 6732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:36405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:36405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:10.198101 6732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:10.199354 6732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:10.210644 6739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:10.210721 6742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:10.210668 6738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:10.211581 6732 server_base.cc:1034] running on GCE node
I20260808 03:52:10.212402 6732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:10.213758 6732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:10.214960 6732 hybrid_clock.cc:648] HybridClock initialized: now 1786161130214912 us; error 40 us; skew 500 ppm
I20260808 03:52:10.215282 6732 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:52:10 dist-test-slave-jlzn krb5kdc[6715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161130, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:52:10.221402 6732 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:52:10.222823 6732 webserver.cc:460] Webserver started at http://127.26.146.190:35701/ using document root <none> and password file <none>
I20260808 03:52:10.223465 6732 fs_manager.cc:359] Metadata directory not provided
I20260808 03:52:10.223580 6732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:52:10.223935 6732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:52:10.226788 6732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "8dd1c5b1538e4697be405a1e4b8d646b"
format_stamp: "Formatted at 2026-08-08 03:52:10 on dist-test-slave-jlzn"
server_key: "bcd5d192470eb562f5da59b54b2242dd"
server_key_iv: "817660510b3371b1586d0689490238c1"
server_key_version: "encryptionkey@0"
I20260808 03:52:10.227506 6732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "8dd1c5b1538e4697be405a1e4b8d646b"
format_stamp: "Formatted at 2026-08-08 03:52:10 on dist-test-slave-jlzn"
server_key: "bcd5d192470eb562f5da59b54b2242dd"
server_key_iv: "817660510b3371b1586d0689490238c1"
server_key_version: "encryptionkey@0"
I20260808 03:52:10.232425 6732 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 03:52:10.235684 6752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:10.236938 6732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:52:10.237092 6732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "8dd1c5b1538e4697be405a1e4b8d646b"
format_stamp: "Formatted at 2026-08-08 03:52:10 on dist-test-slave-jlzn"
server_key: "bcd5d192470eb562f5da59b54b2242dd"
server_key_iv: "817660510b3371b1586d0689490238c1"
server_key_version: "encryptionkey@0"
I20260808 03:52:10.237270 6732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:52:10.291968 6732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:52:10.295972 6732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:52:10.296255 6732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:52:10.313318 6732 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:36405
I20260808 03:52:10.313342 6828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:36405 every 8 connection(s)
I20260808 03:52:10.314627 6732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:52:10.318245 6829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:52:10.319185 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 6732
I20260808 03:52:10.319394 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:52:10.320070 27210 external_mini_cluster.cc:1428] Setting key 96fffbb86d249f48dff0739f610868f7
I20260808 03:52:10.332130 6829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b: Bootstrap starting.
Aug 08 03:52:10 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:52:10.335752 6829 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b: Neither blocks nor log segments found. Creating new log.
I20260808 03:52:10.336830 6829 log.cc:824] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b: Log is configured to *not* fsync() on all Append() calls
I20260808 03:52:10.339986 6829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b: No bootstrap required, opened a new log
I20260808 03:52:10.343014 6835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:10.321946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37844 (local address 127.26.146.190:36405)
0808 03:52:10.322779 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:10.322793 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:10.322838 (+ 45us) server_negotiation.cc:407] Connection header received
0808 03:52:10.324455 (+ 1617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:10.324477 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:10.324840 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:10.325194 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:10.326942 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:10.328256 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:10.329991 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:10.330605 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:10.330741 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:10.335321 (+ 4580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:10.335329 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:10.335343 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:10.335382 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:10.338694 (+ 3312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:10.339327 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:10.339335 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:10.339343 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:10.339562 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:10.339915 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:10.339919 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:10.339922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:10.340637 (+ 715us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:10.340950 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:10.341291 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 03:52:10.341703 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":371,"threads_started":1}
I20260808 03:52:10.348937 6829 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dd1c5b1538e4697be405a1e4b8d646b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36405 } }
I20260808 03:52:10.349274 6829 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:52:10.349349 6829 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dd1c5b1538e4697be405a1e4b8d646b, State: Initialized, Role: FOLLOWER
I20260808 03:52:10.349822 6829 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dd1c5b1538e4697be405a1e4b8d646b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36405 } }
I20260808 03:52:10.350037 6829 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:52:10.350133 6829 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:52:10.350270 6829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:52:10.352788 6829 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dd1c5b1538e4697be405a1e4b8d646b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36405 } }
I20260808 03:52:10.353235 6829 leader_election.cc:302] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8dd1c5b1538e4697be405a1e4b8d646b; no voters:
I20260808 03:52:10.353714 6829 leader_election.cc:288] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:52:10.353837 6840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:52:10.354470 6840 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [term 1 LEADER]: Becoming Leader. State: Replica: 8dd1c5b1538e4697be405a1e4b8d646b, State: Running, Role: LEADER
I20260808 03:52:10.354915 6840 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dd1c5b1538e4697be405a1e4b8d646b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36405 } }
I20260808 03:52:10.355674 6829 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:52:10.357349 6842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8dd1c5b1538e4697be405a1e4b8d646b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dd1c5b1538e4697be405a1e4b8d646b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36405 } } }
I20260808 03:52:10.357960 6842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [sys.catalog]: This master's current role is: LEADER
I20260808 03:52:10.357877 6844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8dd1c5b1538e4697be405a1e4b8d646b. Latest consensus state: current_term: 1 leader_uuid: "8dd1c5b1538e4697be405a1e4b8d646b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dd1c5b1538e4697be405a1e4b8d646b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36405 } } }
I20260808 03:52:10.358196 6844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [sys.catalog]: This master's current role is: LEADER
I20260808 03:52:10.360049 6847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:52:10.365608 6847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:52:10.373986 6847 catalog_manager.cc:1347] Generated new cluster ID: f6ef0d399a6f4ed18646bebffc6acfdd
I20260808 03:52:10.374090 6847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:52:10.382882 6847 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:52:10.384507 6847 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:52:10.394701 6847 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b: Generated new TSK 0
I20260808 03:52:10.395499 6847 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
I20260808 03:52:10.464751 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:36405
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:52:10.796343 6869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:10.796622 6869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:10.796722 6869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:10.804039 6869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:10.804144 6869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:10.804272 6869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:52:10.814281 6869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:36405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:10.815531 6869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:10.816840 6869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:10.829476 6875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:10.829454 6877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:10.829753 6879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:10.830299 6869 server_base.cc:1034] running on GCE node
I20260808 03:52:10.830842 6869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:10.831741 6869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:10.832966 6869 hybrid_clock.cc:648] HybridClock initialized: now 1786161130832921 us; error 37 us; skew 500 ppm
I20260808 03:52:10.833264 6869 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:52:10 dist-test-slave-jlzn krb5kdc[6715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161130, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:52:10.836418 6869 init.cc:377] Logged in from keytab as kudu/127.26.146.129@KRBTEST.COM (short username kudu)
I20260808 03:52:10.837788 6869 webserver.cc:460] Webserver started at http://127.26.146.129:36569/ using document root <none> and password file <none>
I20260808 03:52:10.838435 6869 fs_manager.cc:359] Metadata directory not provided
I20260808 03:52:10.838549 6869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:52:10.838850 6869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:52:10.841907 6869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "378c9377f7b24e2bbbed5c5cf202997d"
format_stamp: "Formatted at 2026-08-08 03:52:10 on dist-test-slave-jlzn"
server_key: "56b901370e69a3659cd15a535b2fc270"
server_key_iv: "4ae96972f001299c6a4d14ba25d2e5c9"
server_key_version: "encryptionkey@0"
I20260808 03:52:10.842912 6869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "378c9377f7b24e2bbbed5c5cf202997d"
format_stamp: "Formatted at 2026-08-08 03:52:10 on dist-test-slave-jlzn"
server_key: "56b901370e69a3659cd15a535b2fc270"
server_key_iv: "4ae96972f001299c6a4d14ba25d2e5c9"
server_key_version: "encryptionkey@0"
I20260808 03:52:10.847930 6869 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 03:52:10.851137 6888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:10.852427 6869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:52:10.852582 6869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "378c9377f7b24e2bbbed5c5cf202997d"
format_stamp: "Formatted at 2026-08-08 03:52:10 on dist-test-slave-jlzn"
server_key: "56b901370e69a3659cd15a535b2fc270"
server_key_iv: "4ae96972f001299c6a4d14ba25d2e5c9"
server_key_version: "encryptionkey@0"
I20260808 03:52:10.852778 6869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:52:10.869165 6869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:52:10.873801 6869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:52:10.874068 6869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:52:10.874996 6869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:52:10.876350 6869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:52:10.876432 6869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:10.876551 6869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:52:10.876608 6869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:10.904220 6869 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:36011
I20260808 03:52:10.904250 7053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:36011 every 8 connection(s)
I20260808 03:52:10.905452 6869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:52:10.910140 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 6869
I20260808 03:52:10.910313 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:52:10.910935 27210 external_mini_cluster.cc:1428] Setting key 7c932b1d2443894fb6fb70797105e85a
Aug 08 03:52:10 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161130, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:52:10.925109 7058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:10.907920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:57749 (local address 127.26.146.190:36405)
0808 03:52:10.908354 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:10.908358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:10.909345 (+ 987us) server_negotiation.cc:407] Connection header received
0808 03:52:10.911247 (+ 1902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:10.911253 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:10.911333 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:10.911429 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:10.913258 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:10.913996 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:10.915538 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:10.915983 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:10.916095 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:10.919411 (+ 3316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:10.919416 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:10.919420 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:10.919453 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:10.921975 (+ 2522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:10.922862 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:10.922867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:10.922869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:10.923000 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:10.923470 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:10.923473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:10.923475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:10.923742 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:10.923891 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:10.924670 (+ 779us) server_negotiation.cc:299] Negotiation successful
0808 03:52:10.924850 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":212,"threads_started":1}
I20260808 03:52:10.926604 7059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:10.908452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.129:57749)
0808 03:52:10.909165 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:10.909207 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:10.910950 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:10.911698 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:10.911707 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:10.912454 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:10.913085 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:10.913103 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:10.914163 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:10.914167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:10.915398 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:10.915412 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:10.916230 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:10.916238 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:10.916457 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:10.917077 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:10.917100 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:10.919204 (+ 2104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:52:10.922118 (+ 2914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:10.922127 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:10.922144 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:10.922735 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:10.923105 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:10.923117 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:10.923120 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:10.923374 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:10.923876 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:52:10.923884 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:52:10.924300 (+ 416us) client_negotiation.cc:769] Sending connection context
0808 03:52:10.924599 (+ 299us) client_negotiation.cc:240] Negotiation successful
0808 03:52:10.925134 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":252,"threads_started":1}
I20260808 03:52:10.928241 7056 heartbeater.cc:347] Connected to a master server at 127.26.146.190:36405
I20260808 03:52:10.928541 7056 heartbeater.cc:464] Registering TS with master...
I20260808 03:52:10.929260 7056 heartbeater.cc:511] Master 127.26.146.190:36405 requested a full tablet report, sending...
I20260808 03:52:10.931306 6778 ts_manager.cc:194] Registered new tserver with Master: 378c9377f7b24e2bbbed5c5cf202997d (127.26.146.129:36011)
I20260808 03:52:10.932837 6778 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.129@KRBTEST.COM'} at 127.26.146.129:57749
WARNING: no policy specified for kudu/127.26.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
I20260808 03:52:10.970803 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:36405
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:52:11.299237 7066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:11.299523 7066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:11.299662 7066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:11.306867 7066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:11.306983 7066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:11.307139 7066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:52:11.319136 7066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:36405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:11.320647 7066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:11.322199 7066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:11.336119 7072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:11.336365 7073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:11.336369 7075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:11.337318 7066 server_base.cc:1034] running on GCE node
I20260808 03:52:11.337867 7066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:11.338776 7066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:11.339959 7066 hybrid_clock.cc:648] HybridClock initialized: now 1786161131339905 us; error 51 us; skew 500 ppm
I20260808 03:52:11.340226 7066 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:52:11 dist-test-slave-jlzn krb5kdc[6715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161131, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:52:11.343271 7066 init.cc:377] Logged in from keytab as kudu/127.26.146.130@KRBTEST.COM (short username kudu)
I20260808 03:52:11.344620 7066 webserver.cc:460] Webserver started at http://127.26.146.130:45643/ using document root <none> and password file <none>
I20260808 03:52:11.345233 7066 fs_manager.cc:359] Metadata directory not provided
I20260808 03:52:11.345328 7066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:52:11.345572 7066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:52:11.348335 7066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "805ef0caaa544538a1bdff3acedf3732"
format_stamp: "Formatted at 2026-08-08 03:52:11 on dist-test-slave-jlzn"
server_key: "068d25081db12a3905fb01c2a3c34d96"
server_key_iv: "ee269814f2d2a08491e6039792bf4937"
server_key_version: "encryptionkey@0"
I20260808 03:52:11.349061 7066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "805ef0caaa544538a1bdff3acedf3732"
format_stamp: "Formatted at 2026-08-08 03:52:11 on dist-test-slave-jlzn"
server_key: "068d25081db12a3905fb01c2a3c34d96"
server_key_iv: "ee269814f2d2a08491e6039792bf4937"
server_key_version: "encryptionkey@0"
I20260808 03:52:11.354050 7066 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:52:11.357563 7085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:11.358809 7066 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 03:52:11.358939 7066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "805ef0caaa544538a1bdff3acedf3732"
format_stamp: "Formatted at 2026-08-08 03:52:11 on dist-test-slave-jlzn"
server_key: "068d25081db12a3905fb01c2a3c34d96"
server_key_iv: "ee269814f2d2a08491e6039792bf4937"
server_key_version: "encryptionkey@0"
I20260808 03:52:11.359107 7066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:52:11.373025 7066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:52:11.376787 7066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:52:11.377029 7066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:52:11.377907 7066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:52:11.379132 7066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:52:11.379204 7066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:11.379297 7066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:52:11.379343 7066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:11.408581 7066 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:34471
I20260808 03:52:11.408620 7249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:34471 every 8 connection(s)
I20260808 03:52:11.409837 7066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:52:11.414976 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 7066
I20260808 03:52:11.415138 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:52:11.415660 27210 external_mini_cluster.cc:1428] Setting key 2ca70f22379b00132fd12be889e967bc
Aug 08 03:52:11 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161131, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:52:11.429317 7058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:11.412253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:41863 (local address 127.26.146.190:36405)
0808 03:52:11.412471 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:11.412478 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:11.413617 (+ 1139us) server_negotiation.cc:407] Connection header received
0808 03:52:11.415413 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:11.415419 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:11.415512 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:11.415614 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:11.417223 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:11.417927 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:11.419416 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:11.419878 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:11.419983 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:11.423563 (+ 3580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:11.423569 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:11.423574 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:11.423616 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:11.426228 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:11.427003 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:11.427007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:11.427010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:11.427139 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:11.427582 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:11.427587 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:11.427591 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:11.427929 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:11.428046 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:11.428849 (+ 803us) server_negotiation.cc:299] Negotiation successful
0808 03:52:11.429014 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 03:52:11.430349 7255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:11.412744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.130:41863)
0808 03:52:11.413454 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:11.413491 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:11.415129 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:11.415857 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:11.415866 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:11.416480 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:11.417042 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:11.417060 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:11.418067 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:11.418070 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:11.419276 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:11.419289 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:11.420099 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:11.420107 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:11.420329 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:11.420971 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:11.420993 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:11.423345 (+ 2352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:52:11.426371 (+ 3026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:11.426379 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:11.426393 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:11.426909 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:11.427238 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:11.427242 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:11.427245 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:11.427472 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:11.428056 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:52:11.428064 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:52:11.428474 (+ 410us) client_negotiation.cc:769] Sending connection context
0808 03:52:11.428790 (+ 316us) client_negotiation.cc:240] Negotiation successful
0808 03:52:11.429283 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":283,"threads_started":1}
I20260808 03:52:11.431787 7250 heartbeater.cc:347] Connected to a master server at 127.26.146.190:36405
I20260808 03:52:11.432126 7250 heartbeater.cc:464] Registering TS with master...
I20260808 03:52:11.433050 7250 heartbeater.cc:511] Master 127.26.146.190:36405 requested a full tablet report, sending...
I20260808 03:52:11.434626 6778 ts_manager.cc:194] Registered new tserver with Master: 805ef0caaa544538a1bdff3acedf3732 (127.26.146.130:34471)
I20260808 03:52:11.435453 6778 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.130@KRBTEST.COM'} at 127.26.146.130:41863
WARNING: no policy specified for kudu/127.26.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
I20260808 03:52:11.473009 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:36405
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:52:11.798497 7258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:11.798799 7258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:11.798954 7258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:11.806420 7258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:11.806545 7258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:11.806702 7258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:52:11.816277 7258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:36405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:11.817539 7258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:11.818802 7258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:11.831338 7270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:11.831290 7273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:11.831362 7271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:11.832165 7258 server_base.cc:1034] running on GCE node
I20260808 03:52:11.832701 7258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:11.833549 7258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:11.834741 7258 hybrid_clock.cc:648] HybridClock initialized: now 1786161131834679 us; error 54 us; skew 500 ppm
I20260808 03:52:11.835042 7258 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:52:11 dist-test-slave-jlzn krb5kdc[6715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161131, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:52:11.838003 7258 init.cc:377] Logged in from keytab as kudu/127.26.146.131@KRBTEST.COM (short username kudu)
I20260808 03:52:11.839365 7258 webserver.cc:460] Webserver started at http://127.26.146.131:33553/ using document root <none> and password file <none>
I20260808 03:52:11.840061 7258 fs_manager.cc:359] Metadata directory not provided
I20260808 03:52:11.840161 7258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:52:11.840416 7258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:52:11.843226 7258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "8df447b7f55a4ec4afe7d3e9a51339c1"
format_stamp: "Formatted at 2026-08-08 03:52:11 on dist-test-slave-jlzn"
server_key: "000c8bc3c0f646d8b872da3c81b26ef5"
server_key_iv: "2f61ab20091d896b25920d116cfbaaed"
server_key_version: "encryptionkey@0"
I20260808 03:52:11.844002 7258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "8df447b7f55a4ec4afe7d3e9a51339c1"
format_stamp: "Formatted at 2026-08-08 03:52:11 on dist-test-slave-jlzn"
server_key: "000c8bc3c0f646d8b872da3c81b26ef5"
server_key_iv: "2f61ab20091d896b25920d116cfbaaed"
server_key_version: "encryptionkey@0"
I20260808 03:52:11.848855 7258 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 03:52:11.852133 7283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:11.853401 7258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 03:52:11.853556 7258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "8df447b7f55a4ec4afe7d3e9a51339c1"
format_stamp: "Formatted at 2026-08-08 03:52:11 on dist-test-slave-jlzn"
server_key: "000c8bc3c0f646d8b872da3c81b26ef5"
server_key_iv: "2f61ab20091d896b25920d116cfbaaed"
server_key_version: "encryptionkey@0"
I20260808 03:52:11.853734 7258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:52:11.866101 7258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:52:11.869796 7258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:52:11.870060 7258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:52:11.870939 7258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:52:11.872292 7258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:52:11.872371 7258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:11.872491 7258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:52:11.872551 7258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:11.902437 7258 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:45815
I20260808 03:52:11.902487 7435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:45815 every 8 connection(s)
I20260808 03:52:11.903687 7258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:52:11.905586 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 7258
I20260808 03:52:11.905758 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:52:11.906359 27210 external_mini_cluster.cc:1428] Setting key 2a26a1e9eadc6cf29258f016ab9844df
Aug 08 03:52:11 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161131, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:52:11.923231 7058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:11.907114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:41605 (local address 127.26.146.190:36405)
0808 03:52:11.907350 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:11.907357 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:11.908078 (+ 721us) server_negotiation.cc:407] Connection header received
0808 03:52:11.910392 (+ 2314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:11.910396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:11.910462 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:11.910606 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:11.912115 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:11.912890 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:11.914321 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:11.914733 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:11.914821 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:11.917961 (+ 3140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:11.917966 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:11.917970 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:11.918003 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:11.920469 (+ 2466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:11.921173 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:11.921177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:11.921179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:11.921335 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:11.921792 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:11.921797 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:11.921801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:11.922067 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:11.922196 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:11.922832 (+ 636us) server_negotiation.cc:299] Negotiation successful
0808 03:52:11.922988 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260808 03:52:11.924299 7439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:11.906675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.131:41605)
0808 03:52:11.907920 (+ 1245us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:11.907972 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:11.910150 (+ 2178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:11.910775 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:11.910782 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:11.911354 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:11.911952 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:11.911972 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:11.913006 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:11.913009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:11.914210 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:11.914222 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:11.914911 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:11.914917 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:11.915102 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:11.915688 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:11.915710 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:11.917776 (+ 2066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:52:11.920598 (+ 2822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:11.920607 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:11.920621 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:11.921055 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:11.921441 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:11.921444 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:11.921447 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:11.921668 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:11.922177 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:52:11.922184 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:52:11.922516 (+ 332us) client_negotiation.cc:769] Sending connection context
0808 03:52:11.922725 (+ 209us) client_negotiation.cc:240] Negotiation successful
0808 03:52:11.923079 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":963,"thread_start_us":767,"threads_started":1}
I20260808 03:52:11.925619 7437 heartbeater.cc:347] Connected to a master server at 127.26.146.190:36405
I20260808 03:52:11.925910 7437 heartbeater.cc:464] Registering TS with master...
I20260808 03:52:11.926575 7437 heartbeater.cc:511] Master 127.26.146.190:36405 requested a full tablet report, sending...
I20260808 03:52:11.927997 6778 ts_manager.cc:194] Registered new tserver with Master: 8df447b7f55a4ec4afe7d3e9a51339c1 (127.26.146.131:45815)
I20260808 03:52:11.928791 6778 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.131@KRBTEST.COM'} at 127.26.146.131:41605
I20260808 03:52:11.933770 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:52:11.935706 7056 heartbeater.cc:503] Master 127.26.146.190:36405 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.26.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.129.keytab.
Entry for principal oryx/127.26.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.129.keytab.
WARNING: no policy specified for oryx/127.26.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.130.keytab.
Entry for principal oryx/127.26.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.130.keytab.
WARNING: no policy specified for oryx/127.26.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.131.keytab.
Entry for principal oryx/127.26.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.131.keytab.
I20260808 03:52:12.082016 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 6869
Aug 08 03:52:12 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.129@KRBTEST.COM
I20260808 03:52:12.097842 7451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:12.083186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55320 (local address 127.26.146.129:36011)
0808 03:52:12.083875 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:12.083882 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:12.083907 (+ 25us) server_negotiation.cc:407] Connection header received
0808 03:52:12.083970 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:12.083977 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:12.084191 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:12.084331 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:12.085714 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:12.086713 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:12.087959 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:12.088517 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:12.088632 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:12.092659 (+ 4027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:12.092672 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:12.092687 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:12.092725 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:12.095439 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:12.096118 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:12.096128 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:12.096131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:12.096272 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:12.096564 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:12.096567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:12.096569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:12.096913 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:12.097138 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:12.097322 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 03:52:12.097526 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":289,"threads_started":1}
I20260808 03:52:12.099066 7048 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:12.183079 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 6869
I20260808 03:52:12.205006 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 7066
Aug 08 03:52:12 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.130@KRBTEST.COM
I20260808 03:52:12.219769 7454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:12.205871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33848 (local address 127.26.146.130:34471)
0808 03:52:12.206417 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:12.206424 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:12.206450 (+ 26us) server_negotiation.cc:407] Connection header received
0808 03:52:12.206514 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:12.206520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:12.206762 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:12.206889 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:12.208113 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:12.209147 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:12.210340 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:12.210864 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:12.210965 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:12.214981 (+ 4016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:12.214992 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:12.215006 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:12.215040 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:12.217576 (+ 2536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:12.218140 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:12.218146 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:12.218149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:12.218261 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:12.218527 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:12.218530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:12.218532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:12.218860 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:12.219111 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:12.219290 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 03:52:12.219479 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":284,"threads_started":1}
I20260808 03:52:12.221101 7241 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:12.301280 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 7066
I20260808 03:52:12.320766 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 7258
Aug 08 03:52:12 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.131@KRBTEST.COM
I20260808 03:52:12.336239 7456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:12.321756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60802 (local address 127.26.146.131:45815)
0808 03:52:12.322387 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:12.322395 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:12.322423 (+ 28us) server_negotiation.cc:407] Connection header received
0808 03:52:12.322480 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:12.322486 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:12.322687 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:12.322820 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:12.324233 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:12.325281 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:12.326516 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:12.327044 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:12.327135 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:12.331208 (+ 4073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:12.331220 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:12.331235 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:12.331264 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:12.333912 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:12.334503 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:12.334510 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:12.334512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:12.334639 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:12.334925 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:12.334928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:12.334930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:12.335251 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:12.335491 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:12.335677 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 03:52:12.335948 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":337,"threads_started":1}
I20260808 03:52:12.337528 7428 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:12.418102 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 7258
I20260808 03:52:12.438951 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 6732
I20260808 03:52:12.440136 6823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:12.542593 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 6732
I20260808 03:52:12.571219 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:36405
--webserver_interface=127.26.146.190
--webserver_port=35701
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:36405
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:52:12.905280 7459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:12.905593 7459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:12.905730 7459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:12.912935 7459 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:52:12.913044 7459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:12.913122 7459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:12.913203 7459 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:52:12.913262 7459 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:52:12.922966 7459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:36405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:36405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=35701
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:12.924450 7459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:12.925812 7459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:12.937239 7468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:12.937217 7466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:12.937544 7464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:12.938083 7459 server_base.cc:1034] running on GCE node
I20260808 03:52:12.938822 7459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:12.940182 7459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:12.941399 7459 hybrid_clock.cc:648] HybridClock initialized: now 1786161132941343 us; error 42 us; skew 500 ppm
I20260808 03:52:12.941804 7459 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:52:12 dist-test-slave-jlzn krb5kdc[6715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161132, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:52:12.945264 7459 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:52:12.946847 7459 webserver.cc:460] Webserver started at http://127.26.146.190:35701/ using document root <none> and password file <none>
I20260808 03:52:12.947510 7459 fs_manager.cc:359] Metadata directory not provided
I20260808 03:52:12.947654 7459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:52:12.953243 7459 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 03:52:12.956101 7479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:12.957386 7459 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 03:52:12.957554 7459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "8dd1c5b1538e4697be405a1e4b8d646b"
format_stamp: "Formatted at 2026-08-08 03:52:10 on dist-test-slave-jlzn"
server_key: "bcd5d192470eb562f5da59b54b2242dd"
server_key_iv: "817660510b3371b1586d0689490238c1"
server_key_version: "encryptionkey@0"
I20260808 03:52:12.958521 7459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:52:12.983371 7459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:52:12.987447 7459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:52:12.987690 7459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:52:13.005141 7459 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:36405
I20260808 03:52:13.005174 7552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:36405 every 8 connection(s)
I20260808 03:52:13.006865 7459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:52:13.013602 7553 sys_catalog.cc:263] Verifying existing consensus state
I20260808 03:52:13.015045 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 7459
I20260808 03:52:13.016115 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:36011
--local_ip_for_outbound_sockets=127.26.146.129
--tserver_master_addrs=127.26.146.190:36405
--webserver_port=36569
--webserver_interface=127.26.146.129
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.26.146.129
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:52:13.017805 7553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b: Bootstrap starting.
I20260808 03:52:13.038882 7553 log.cc:824] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b: Log is configured to *not* fsync() on all Append() calls
I20260808 03:52:13.047992 7553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 03:52:13.048506 7553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b: Bootstrap complete.
I20260808 03:52:13.057502 7553 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dd1c5b1538e4697be405a1e4b8d646b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36405 } }
I20260808 03:52:13.058303 7553 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dd1c5b1538e4697be405a1e4b8d646b, State: Initialized, Role: FOLLOWER
I20260808 03:52:13.058923 7553 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dd1c5b1538e4697be405a1e4b8d646b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36405 } }
I20260808 03:52:13.059154 7553 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:52:13.059268 7553 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:52:13.059435 7553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [term 1 FOLLOWER]: Advancing to term 2
I20260808 03:52:13.063226 7553 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dd1c5b1538e4697be405a1e4b8d646b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36405 } }
I20260808 03:52:13.063694 7553 leader_election.cc:302] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8dd1c5b1538e4697be405a1e4b8d646b; no voters:
I20260808 03:52:13.064371 7553 leader_election.cc:288] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 03:52:13.064505 7563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [term 2 FOLLOWER]: Leader election won for term 2
I20260808 03:52:13.066496 7553 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:52:13.071496 7563 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [term 2 LEADER]: Becoming Leader. State: Replica: 8dd1c5b1538e4697be405a1e4b8d646b, State: Running, Role: LEADER
I20260808 03:52:13.072242 7563 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dd1c5b1538e4697be405a1e4b8d646b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36405 } }
I20260808 03:52:13.074771 7564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8dd1c5b1538e4697be405a1e4b8d646b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dd1c5b1538e4697be405a1e4b8d646b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36405 } } }
I20260808 03:52:13.075141 7564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [sys.catalog]: This master's current role is: LEADER
I20260808 03:52:13.075659 7565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8dd1c5b1538e4697be405a1e4b8d646b. Latest consensus state: current_term: 2 leader_uuid: "8dd1c5b1538e4697be405a1e4b8d646b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dd1c5b1538e4697be405a1e4b8d646b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36405 } } }
I20260808 03:52:13.076094 7565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b [sys.catalog]: This master's current role is: LEADER
I20260808 03:52:13.097208 7579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:52:13.107646 7579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:52:13.109194 7579 catalog_manager.cc:1259] Loaded cluster ID: f6ef0d399a6f4ed18646bebffc6acfdd
I20260808 03:52:13.109944 7579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:52:13.113967 7579 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:52:13.115432 7579 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8dd1c5b1538e4697be405a1e4b8d646b: Loaded TSK: 0
I20260808 03:52:13.116943 7579 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 03:52:13.363588 7560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:13.363921 7560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:13.364071 7560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:13.371184 7560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:13.371285 7560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:13.371426 7560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:52:13.381125 7560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.129.keytab
--principal=oryx/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:36011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=36569
--tserver_master_addrs=127.26.146.190:36405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:13.382469 7560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:13.383782 7560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:13.397020 7597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:13.397297 7600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:13.397832 7596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:13.398550 7560 server_base.cc:1034] running on GCE node
I20260808 03:52:13.399142 7560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:13.400130 7560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:13.401348 7560 hybrid_clock.cc:648] HybridClock initialized: now 1786161133401290 us; error 49 us; skew 500 ppm
I20260808 03:52:13.401638 7560 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:52:13 dist-test-slave-jlzn krb5kdc[6715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161133, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:52:13.404825 7560 init.cc:377] Logged in from keytab as oryx/127.26.146.129@KRBTEST.COM (short username oryx)
I20260808 03:52:13.406265 7560 webserver.cc:460] Webserver started at http://127.26.146.129:36569/ using document root <none> and password file <none>
I20260808 03:52:13.406915 7560 fs_manager.cc:359] Metadata directory not provided
I20260808 03:52:13.407045 7560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:52:13.412341 7560 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 03:52:13.415005 7607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:13.416414 7560 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:52:13.416563 7560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "378c9377f7b24e2bbbed5c5cf202997d"
format_stamp: "Formatted at 2026-08-08 03:52:10 on dist-test-slave-jlzn"
server_key: "56b901370e69a3659cd15a535b2fc270"
server_key_iv: "4ae96972f001299c6a4d14ba25d2e5c9"
server_key_version: "encryptionkey@0"
I20260808 03:52:13.417472 7560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:52:13.440399 7560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:52:13.444432 7560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:52:13.444687 7560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:52:13.445617 7560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:52:13.446928 7560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:52:13.447031 7560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:13.447134 7560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:52:13.447202 7560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:13.477877 7560 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:36011
I20260808 03:52:13.477938 7760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:36011 every 8 connection(s)
I20260808 03:52:13.479334 7560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:52:13.483753 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 7560
I20260808 03:52:13.484400 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:34471
--local_ip_for_outbound_sockets=127.26.146.130
--tserver_master_addrs=127.26.146.190:36405
--webserver_port=45643
--webserver_interface=127.26.146.130
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.26.146.130
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 03:52:13 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:13 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:13.497326 7764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:13.482191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.129:52627)
0808 03:52:13.482977 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:13.483023 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:13.484843 (+ 1820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:13.486408 (+ 1565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:13.486415 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:13.486940 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:13.487462 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:13.487479 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:13.489594 (+ 2115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:13.489598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:13.490791 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:13.490803 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:13.491871 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:13.491880 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:13.492124 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:13.492748 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:13.492770 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:13.496398 (+ 3628us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":306,"threads_started":1}
W20260808 03:52:13.497933 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:13.482149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:52627 (local address 127.26.146.190:36405)
0808 03:52:13.482816 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:13.482828 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:13.483132 (+ 304us) server_negotiation.cc:407] Connection header received
0808 03:52:13.485443 (+ 2311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:13.485464 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:13.485911 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:13.486322 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:13.487665 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:13.489419 (+ 1754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:13.490938 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:13.491609 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:13.491777 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:13.496799 (+ 5022us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.129:52627: BlockingRecv error: recv got EOF from 127.26.146.129:52627 (error 108)
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":236,"threads_started":1}
W20260808 03:52:13.498122 7761 heartbeater.cc:644] Failed to heartbeat to 127.26.146.190:36405 (0 consecutive failures): Not authorized: Failed to ping master at 127.26.146.190:36405: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Aug 08 03:52:13 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:13 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:13.506515 7764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:13.498684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.129:35717)
0808 03:52:13.498871 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:13.498888 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:13.498986 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:13.499266 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:13.499270 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:13.499659 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:13.499857 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:13.499870 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:13.500832 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:13.500837 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:13.502100 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:13.502116 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:13.502716 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:13.502721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:13.502882 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:13.503413 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:13.503434 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:13.506254 (+ 2820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 03:52:13.506671 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:13.498885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:35717 (local address 127.26.146.190:36405)
0808 03:52:13.499024 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:13.499028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:13.499044 (+ 16us) server_negotiation.cc:407] Connection header received
0808 03:52:13.499088 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:13.499091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:13.499144 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:13.499226 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:13.499985 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:13.500693 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:13.502219 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:13.502575 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:13.502656 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:13.506438 (+ 3782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.129:35717: BlockingRecv error: recv got EOF from 127.26.146.129:35717 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 03:52:13 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:13 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:13.514393 7764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:13.507372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.129:40453)
0808 03:52:13.507528 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:13.507546 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:13.507639 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:13.508013 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:13.508018 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:13.508303 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:13.508438 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:13.508447 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:13.509415 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:13.509418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:13.510292 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:13.510303 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:13.510947 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:13.510953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:13.511103 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:13.511654 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:13.511669 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:13.514104 (+ 2435us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 03:52:13.514549 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:13.507432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:40453 (local address 127.26.146.190:36405)
0808 03:52:13.507571 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:13.507576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:13.507592 (+ 16us) server_negotiation.cc:407] Connection header received
0808 03:52:13.507765 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:13.507770 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:13.507884 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:13.507960 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:13.508567 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:13.509280 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:13.510420 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:13.510821 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:13.510921 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:13.514317 (+ 3396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.129:40453: BlockingRecv error: recv got EOF from 127.26.146.129:40453 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 03:52:13.514848 7761 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 03:52:13.819685 7765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:13.820004 7765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:13.820108 7765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:13.827322 7765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:13.827430 7765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:13.827556 7765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:52:13.837136 7765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.130.keytab
--principal=oryx/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:34471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=45643
--tserver_master_addrs=127.26.146.190:36405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:13.838367 7765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:13.839627 7765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:13.852048 7773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:13.852083 7776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:13.852110 7780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:13.852818 7765 server_base.cc:1034] running on GCE node
I20260808 03:52:13.853372 7765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:13.854228 7765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:13.855424 7765 hybrid_clock.cc:648] HybridClock initialized: now 1786161133855385 us; error 34 us; skew 500 ppm
I20260808 03:52:13.855713 7765 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:52:13 dist-test-slave-jlzn krb5kdc[6715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161133, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:52:13.858624 7765 init.cc:377] Logged in from keytab as oryx/127.26.146.130@KRBTEST.COM (short username oryx)
I20260808 03:52:13.860024 7765 webserver.cc:460] Webserver started at http://127.26.146.130:45643/ using document root <none> and password file <none>
I20260808 03:52:13.860636 7765 fs_manager.cc:359] Metadata directory not provided
I20260808 03:52:13.860756 7765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:52:13.866341 7765 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 03:52:13.868979 7794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:13.870247 7765 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 03:52:13.870396 7765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "805ef0caaa544538a1bdff3acedf3732"
format_stamp: "Formatted at 2026-08-08 03:52:11 on dist-test-slave-jlzn"
server_key: "068d25081db12a3905fb01c2a3c34d96"
server_key_iv: "ee269814f2d2a08491e6039792bf4937"
server_key_version: "encryptionkey@0"
I20260808 03:52:13.871335 7765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:52:13.892786 7765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:52:13.896482 7765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:52:13.896731 7765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:52:13.897679 7765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:52:13.898955 7765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:52:13.899050 7765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:13.899161 7765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:52:13.899233 7765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:13.929064 7765 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:34471
I20260808 03:52:13.929134 7953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:34471 every 8 connection(s)
I20260808 03:52:13.930444 7765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:52:13.933629 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 7765
I20260808 03:52:13.934377 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:45815
--local_ip_for_outbound_sockets=127.26.146.131
--tserver_master_addrs=127.26.146.190:36405
--webserver_port=33553
--webserver_interface=127.26.146.131
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.26.146.131
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 03:52:13 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:13 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 03:52:13.946027 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:13.932954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:49251 (local address 127.26.146.190:36405)
0808 03:52:13.933139 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:13.933144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:13.934414 (+ 1270us) server_negotiation.cc:407] Connection header received
0808 03:52:13.936324 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:13.936329 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:13.936408 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:13.936495 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:13.937979 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:13.938691 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:13.940366 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:13.940924 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:13.941054 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:13.945832 (+ 4778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.130:49251: BlockingRecv error: recv got EOF from 127.26.146.130:49251 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 03:52:13.946851 7958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:13.933482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.130:49251)
0808 03:52:13.934257 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:13.934302 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:13.936064 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:13.936740 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:13.936748 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:13.937268 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:13.937812 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:13.937830 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:13.938841 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:13.938845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:13.940191 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:13.940206 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:13.941173 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:13.941180 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:13.941430 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:13.942133 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:13.942161 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:13.945675 (+ 3514us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":320,"threads_started":1}
W20260808 03:52:13.947486 7954 heartbeater.cc:644] Failed to heartbeat to 127.26.146.190:36405 (0 consecutive failures): Not authorized: Failed to ping master at 127.26.146.190:36405: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Aug 08 03:52:13 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:13 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:13.955787 7958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:13.948076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.130:49147)
0808 03:52:13.948241 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:13.948258 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:13.948350 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:13.948628 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:13.948632 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:13.949002 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:13.949189 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:13.949203 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:13.950107 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:13.950119 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:13.951366 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:13.951380 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:13.952056 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:13.952061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:13.952225 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:13.952735 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:13.952754 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:13.955526 (+ 2772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 03:52:13.956008 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:13.948187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:49147 (local address 127.26.146.190:36405)
0808 03:52:13.948341 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:13.948345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:13.948361 (+ 16us) server_negotiation.cc:407] Connection header received
0808 03:52:13.948457 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:13.948461 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:13.948515 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:13.948582 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:13.949310 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:13.949975 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:13.951479 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:13.951918 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:13.952011 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:13.955763 (+ 3752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.130:49147: BlockingRecv error: recv got EOF from 127.26.146.130:49147 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 03:52:13 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:13 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:13.964226 7958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:13.956535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.130:38177)
0808 03:52:13.956752 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:13.956769 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:13.956844 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:13.957160 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:13.957163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:13.957431 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:13.957573 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:13.957580 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:13.958868 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:13.958872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:13.959747 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:13.959757 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:13.960636 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:13.960640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:13.960758 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:13.961268 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:13.961293 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:13.964009 (+ 2716us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122}
W20260808 03:52:13.964494 7954 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 03:52:13.964768 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:13.956676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:38177 (local address 127.26.146.190:36405)
0808 03:52:13.956867 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:13.956873 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:13.956912 (+ 39us) server_negotiation.cc:407] Connection header received
0808 03:52:13.956967 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:13.956972 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:13.957044 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:13.957145 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:13.957700 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:13.958738 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:13.959893 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:13.960511 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:13.960636 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:13.964252 (+ 3616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.130:38177: BlockingRecv error: recv got EOF from 127.26.146.130:38177 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 03:52:14.265619 7959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:14.265882 7959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:14.265983 7959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:14.273255 7959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:14.273362 7959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:14.273475 7959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:52:14.282975 7959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/krb5kdc/oryx_127.26.146.131.keytab
--principal=oryx/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:45815
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=33553
--tserver_master_addrs=127.26.146.190:36405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:14.284216 7959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:14.285456 7959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:14.298008 7968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:14.297914 7971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:14.298040 7973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:14.299007 7959 server_base.cc:1034] running on GCE node
I20260808 03:52:14.299603 7959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:14.300541 7959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:14.301750 7959 hybrid_clock.cc:648] HybridClock initialized: now 1786161134301693 us; error 46 us; skew 500 ppm
I20260808 03:52:14.302043 7959 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:52:14 dist-test-slave-jlzn krb5kdc[6715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161134, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:52:14.305035 7959 init.cc:377] Logged in from keytab as oryx/127.26.146.131@KRBTEST.COM (short username oryx)
I20260808 03:52:14.306416 7959 webserver.cc:460] Webserver started at http://127.26.146.131:33553/ using document root <none> and password file <none>
I20260808 03:52:14.307053 7959 fs_manager.cc:359] Metadata directory not provided
I20260808 03:52:14.307191 7959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:52:14.312356 7959 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 03:52:14.314841 7985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:14.316092 7959 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:52:14.316253 7959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "8df447b7f55a4ec4afe7d3e9a51339c1"
format_stamp: "Formatted at 2026-08-08 03:52:11 on dist-test-slave-jlzn"
server_key: "000c8bc3c0f646d8b872da3c81b26ef5"
server_key_iv: "2f61ab20091d896b25920d116cfbaaed"
server_key_version: "encryptionkey@0"
I20260808 03:52:14.317163 7959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:52:14.339128 7959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:52:14.342701 7959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:52:14.342937 7959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:52:14.343886 7959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:52:14.345209 7959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:52:14.345321 7959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:14.345428 7959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:52:14.345505 7959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:14.375130 7959 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:45815
I20260808 03:52:14.375192 8138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:45815 every 8 connection(s)
I20260808 03:52:14.376605 7959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:52:14.381842 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 7959
Aug 08 03:52:14 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:14 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 03:52:14.393076 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:14.378911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:36953 (local address 127.26.146.190:36405)
0808 03:52:14.379085 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:14.379091 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:14.380242 (+ 1151us) server_negotiation.cc:407] Connection header received
0808 03:52:14.382168 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:14.382173 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:14.382243 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:14.382327 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:14.384253 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:14.385065 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:14.386876 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:14.387277 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:14.387364 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:14.392799 (+ 5435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.131:36953: BlockingRecv error: recv got EOF from 127.26.146.131:36953 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 03:52:14.393362 8144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:14.379371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.131:36953)
0808 03:52:14.380079 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:14.380127 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:14.381881 (+ 1754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:14.382564 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:14.382572 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:14.383256 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:14.384100 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:14.384121 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:14.385222 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:14.385229 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:14.386738 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:14.386750 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:14.387490 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:14.387497 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:14.387774 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:14.388465 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:14.388492 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:14.392304 (+ 3812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":258,"threads_started":1}
W20260808 03:52:14.393961 8141 heartbeater.cc:644] Failed to heartbeat to 127.26.146.190:36405 (0 consecutive failures): Not authorized: Failed to ping master at 127.26.146.190:36405: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
I20260808 03:52:14.398193 8147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:14.382990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47900 (local address 127.26.146.190:36405)
0808 03:52:14.384146 (+ 1156us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:14.384150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:14.384165 (+ 15us) server_negotiation.cc:407] Connection header received
0808 03:52:14.384214 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:14.384219 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:14.384293 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:14.384410 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:14.386063 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:14.386817 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:14.388334 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:14.388743 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:14.388839 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:14.391760 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:14.391768 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:14.391782 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:14.391864 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:14.395195 (+ 3331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:14.395825 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:14.395834 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:14.395846 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:14.396023 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:14.396382 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:14.396387 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:14.396391 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:14.396832 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:14.397101 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:14.397545 (+ 444us) server_negotiation.cc:299] Negotiation successful
0808 03:52:14.397903 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1058,"thread_start_us":951,"threads_started":1}
Aug 08 03:52:14 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:14 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:14.401801 8144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:14.394465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.131:59427)
0808 03:52:14.394655 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:14.394674 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:14.394768 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:14.395036 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:14.395041 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:14.395398 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:14.395528 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:14.395535 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:14.396528 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:14.396534 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:14.397381 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:14.397392 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:14.398160 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:14.398165 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:14.398333 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:14.398787 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:14.398807 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:14.401542 (+ 2735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 03:52:14.401932 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:14.394577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:59427 (local address 127.26.146.190:36405)
0808 03:52:14.394704 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:14.394708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:14.394723 (+ 15us) server_negotiation.cc:407] Connection header received
0808 03:52:14.394867 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:14.394870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:14.394918 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:14.394992 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:14.395656 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:14.396402 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:14.397511 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:14.398025 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:14.398138 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:14.401731 (+ 3593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.131:59427: BlockingRecv error: recv got EOF from 127.26.146.131:59427 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 08 03:52:14 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:14 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:14.408602 8144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:14.402335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.131:47877)
0808 03:52:14.402473 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:14.402491 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:14.402582 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:14.402843 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:14.402846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:14.403084 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:14.403212 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:14.403219 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:14.404125 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:14.404129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:14.404988 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:14.404998 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:14.405544 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:14.405548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:14.405668 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:14.406069 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:14.406083 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:14.408417 (+ 2334us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 03:52:14.408849 8141 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 03:52:14.408872 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:14.402439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:47877 (local address 127.26.146.190:36405)
0808 03:52:14.402643 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:14.402646 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:14.402659 (+ 13us) server_negotiation.cc:407] Connection header received
0808 03:52:14.402693 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:14.402695 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:14.402751 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:14.402820 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:14.403305 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:14.404022 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:14.405082 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:14.405449 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:14.405529 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:14.408570 (+ 3041us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.131:47877: BlockingRecv error: recv got EOF from 127.26.146.131:47877 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
Aug 08 03:52:14 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:14 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:14.523711 8148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:14.515411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.129:42063)
0808 03:52:14.515914 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:14.515930 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:14.516049 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:14.516324 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:14.516328 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:14.516792 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:14.516990 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:14.517000 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:14.518026 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:14.518030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:14.518983 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:14.518997 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:14.519749 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:14.519755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:14.519937 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:14.520531 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:14.520552 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:14.523393 (+ 2841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":241,"threads_started":1}
W20260808 03:52:14.523865 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:14.515496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:42063 (local address 127.26.146.190:36405)
0808 03:52:14.515732 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:14.515739 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:14.515999 (+ 260us) server_negotiation.cc:407] Connection header received
0808 03:52:14.516141 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:14.516144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:14.516217 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:14.516293 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:14.517159 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:14.517880 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:14.519135 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:14.519576 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:14.519687 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:14.523630 (+ 3943us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.129:42063: BlockingRecv error: recv got EOF from 127.26.146.129:42063 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 08 03:52:14 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:14 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:14.973126 8149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:14.965046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.130:37225)
0808 03:52:14.965491 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:14.965507 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:14.965626 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:14.965903 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:14.965906 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:14.966379 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:14.966597 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:14.966606 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:14.967526 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:14.967531 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:14.968574 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:14.968587 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:14.969252 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:14.969257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:14.969374 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:14.969959 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:14.969978 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:14.972856 (+ 2878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":227,"threads_started":1}
W20260808 03:52:14.973311 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:14.965112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:37225 (local address 127.26.146.190:36405)
0808 03:52:14.965324 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:14.965330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:14.965576 (+ 246us) server_negotiation.cc:407] Connection header received
0808 03:52:14.965738 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:14.965742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:14.965804 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:14.965875 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:14.966707 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:14.967400 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:14.968694 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:14.969117 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:14.969209 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:14.973049 (+ 3840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.130:37225: BlockingRecv error: recv got EOF from 127.26.146.130:37225 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 08 03:52:15 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:15 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:15.419827 8150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:15.409376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.131:43687)
0808 03:52:15.409849 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:15.409869 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:15.409981 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:15.410262 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:15.410265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:15.410702 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:15.410898 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:15.410907 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:15.411858 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:15.411861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:15.412789 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:15.412800 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:15.413416 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:15.413420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:15.413537 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:15.416957 (+ 3420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:15.416975 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:15.419577 (+ 2602us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":239,"threads_started":1}
W20260808 03:52:15.420040 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:15.409479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:43687 (local address 127.26.146.190:36405)
0808 03:52:15.409661 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:15.409667 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:15.409933 (+ 266us) server_negotiation.cc:407] Connection header received
0808 03:52:15.410066 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:15.410069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:15.410135 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:15.410214 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:15.411008 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:15.411737 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:15.412899 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:15.413286 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:15.413375 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:15.419784 (+ 6409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.131:43687: BlockingRecv error: recv got EOF from 127.26.146.131:43687 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 03:52:15 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:15 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:15.533108 8151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:15.524591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.129:36045)
0808 03:52:15.525094 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:15.525127 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:15.525257 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:15.525576 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:15.525582 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:15.526029 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:15.526209 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:15.526219 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:15.527266 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:15.527271 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:15.528238 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:15.528253 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:15.529011 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:15.529017 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:15.529178 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:15.529786 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:15.529805 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:15.532860 (+ 3055us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":271,"threads_started":1}
W20260808 03:52:15.533279 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:15.524671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:36045 (local address 127.26.146.190:36405)
0808 03:52:15.524862 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:15.524869 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:15.525203 (+ 334us) server_negotiation.cc:407] Connection header received
0808 03:52:15.525382 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:15.525386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:15.525446 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:15.525524 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:15.526347 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:15.527132 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:15.528381 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:15.528862 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:15.528971 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:15.533037 (+ 4066us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.129:36045: BlockingRecv error: recv got EOF from 127.26.146.129:36045 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 03:52:15 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:15 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:15.982020 8152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:15.974014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.130:52855)
0808 03:52:15.974538 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:15.974554 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:15.974686 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:15.975013 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:15.975018 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:15.975482 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:15.975728 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:15.975740 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:15.976869 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:15.976874 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:15.977841 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:15.977855 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:15.978497 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:15.978501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:15.978610 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:15.979135 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:15.979152 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:15.981780 (+ 2628us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":285,"threads_started":1}
W20260808 03:52:15.982159 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:15.974094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:52855 (local address 127.26.146.190:36405)
0808 03:52:15.974290 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:15.974297 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:15.974649 (+ 352us) server_negotiation.cc:407] Connection header received
0808 03:52:15.974825 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:15.974829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:15.974890 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:15.974964 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:15.975879 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:15.976732 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:15.977972 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:15.978377 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:15.978470 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:15.981967 (+ 3497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.130:52855: BlockingRecv error: recv got EOF from 127.26.146.130:52855 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 03:52:16 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:16 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:16.428519 8154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:16.420635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.131:35003)
0808 03:52:16.421196 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:16.421217 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:16.421329 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:16.421643 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:16.421646 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:16.422068 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:16.422281 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:16.422296 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:16.423250 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:16.423254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:16.424176 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:16.424187 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:16.424779 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:16.424784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:16.424887 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:16.425442 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:16.425460 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:16.428273 (+ 2813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":323,"threads_started":1}
W20260808 03:52:16.428711 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:16.420763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:35003 (local address 127.26.146.190:36405)
0808 03:52:16.420907 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:16.420912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:16.421281 (+ 369us) server_negotiation.cc:407] Connection header received
0808 03:52:16.421413 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:16.421417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:16.421499 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:16.421578 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:16.422410 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:16.423146 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:16.424272 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:16.424665 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:16.424755 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:16.428456 (+ 3701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.131:35003: BlockingRecv error: recv got EOF from 127.26.146.131:35003 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 03:52:16 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:16 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:16.541975 8155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:16.534005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.129:50363)
0808 03:52:16.534601 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:16.534619 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:16.534765 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:16.535085 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:16.535088 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:16.535426 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:16.535603 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:16.535613 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:16.536694 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:16.536698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:16.537611 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:16.537625 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:16.538300 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:16.538304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:16.538430 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:16.539011 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:16.539038 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:16.541726 (+ 2688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":344,"threads_started":1}
W20260808 03:52:16.542119 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:16.534008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:50363 (local address 127.26.146.190:36405)
0808 03:52:16.534207 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:16.534211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:16.534704 (+ 493us) server_negotiation.cc:407] Connection header received
0808 03:52:16.534885 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:16.534889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:16.534951 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:16.535025 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:16.535717 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:16.536560 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:16.537765 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:16.538165 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:16.538270 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:16.541898 (+ 3628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.129:50363: BlockingRecv error: recv got EOF from 127.26.146.129:50363 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 08 03:52:16 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:16 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:16.990842 8156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:16.983030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.130:58973)
0808 03:52:16.983550 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:16.983567 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:16.983687 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:16.984001 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:16.984006 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:16.984393 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:16.984569 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:16.984579 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:16.985534 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:16.985539 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:16.986433 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:16.986446 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:16.987031 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:16.987035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:16.987156 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:16.987795 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:16.987846 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:16.990585 (+ 2739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":268,"threads_started":1}
W20260808 03:52:16.990906 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:16.983094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:58973 (local address 127.26.146.190:36405)
0808 03:52:16.983265 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:16.983270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:16.983632 (+ 362us) server_negotiation.cc:407] Connection header received
0808 03:52:16.983792 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:16.983796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:16.983883 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:16.983970 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:16.984688 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:16.985431 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:16.986539 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:16.986910 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:16.986993 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:16.990717 (+ 3724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.130:58973: BlockingRecv error: recv got EOF from 127.26.146.130:58973 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 03:52:17 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:17 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:17.437541 8158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:17.429374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.131:55801)
0808 03:52:17.429907 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:17.429926 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:17.430056 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:17.430403 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:17.430408 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:17.430787 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:17.430965 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:17.430975 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:17.432083 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:17.432088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:17.433026 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:17.433040 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:17.433873 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:17.433880 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:17.434014 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:17.434680 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:17.434703 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:17.437292 (+ 2589us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":289,"threads_started":1}
W20260808 03:52:17.437659 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:17.429514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:55801 (local address 127.26.146.190:36405)
0808 03:52:17.429736 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:17.429742 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:17.430007 (+ 265us) server_negotiation.cc:407] Connection header received
0808 03:52:17.430200 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:17.430204 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:17.430266 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:17.430349 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:17.431120 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:17.431910 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:17.433170 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:17.433726 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:17.433844 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:17.437478 (+ 3634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.131:55801: BlockingRecv error: recv got EOF from 127.26.146.131:55801 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 08 03:52:17 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:17 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:17.551148 8159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:17.542783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.129:55657)
0808 03:52:17.543283 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:17.543299 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:17.543428 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:17.543752 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:17.543756 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:17.544185 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:17.544342 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:17.544352 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:17.545347 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:17.545351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:17.546326 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:17.546339 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:17.547011 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:17.547017 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:17.547183 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:17.547748 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:17.547767 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:17.550909 (+ 3142us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":273,"threads_started":1}
W20260808 03:52:17.551350 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:17.542858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:55657 (local address 127.26.146.190:36405)
0808 03:52:17.543064 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:17.543071 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:17.543376 (+ 305us) server_negotiation.cc:407] Connection header received
0808 03:52:17.543540 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:17.543547 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:17.543617 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:17.543695 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:17.544480 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:17.545218 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:17.546460 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:17.546875 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:17.546982 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:17.551103 (+ 4121us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.129:55657: BlockingRecv error: recv got EOF from 127.26.146.129:55657 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 03:52:17 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:17 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:17.999934 8160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:17.991759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.130:40787)
0808 03:52:17.992234 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:17.992250 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:17.992392 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:17.992738 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:17.992742 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:17.993242 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:17.993409 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:17.993421 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:17.994590 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:17.994594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:17.995543 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:17.995554 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:17.996163 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:17.996167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:17.996286 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:17.996898 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:17.996916 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:17.999672 (+ 2756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":224,"threads_started":1}
W20260808 03:52:18.000097 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:17.991855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:40787 (local address 127.26.146.190:36405)
0808 03:52:17.992063 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:17.992069 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:17.992327 (+ 258us) server_negotiation.cc:407] Connection header received
0808 03:52:17.992502 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:17.992505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:17.992582 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:17.992668 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:17.993568 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:17.994468 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:17.995650 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:17.996061 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:17.996147 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:17.999895 (+ 3748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.130:40787: BlockingRecv error: recv got EOF from 127.26.146.130:40787 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 03:52:18 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:18 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:18.446221 8161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:18.438426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.131:43479)
0808 03:52:18.438910 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:18.438931 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:18.439046 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:18.439316 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:18.439319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:18.439699 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:18.439882 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:18.439892 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:18.440978 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:18.440982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:18.441945 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:18.441958 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:18.442687 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:18.442691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:18.442811 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:18.443352 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:18.443369 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:18.445993 (+ 2624us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":258,"threads_started":1}
W20260808 03:52:18.446393 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:18.438488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:43479 (local address 127.26.146.190:36405)
0808 03:52:18.438678 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:18.438682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:18.439001 (+ 319us) server_negotiation.cc:407] Connection header received
0808 03:52:18.439139 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:18.439142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:18.439204 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:18.439282 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:18.440047 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:18.440872 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:18.442083 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:18.442564 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:18.442669 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:18.446177 (+ 3508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.131:43479: BlockingRecv error: recv got EOF from 127.26.146.131:43479 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 03:52:18 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:18 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:18.560144 8162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:18.552078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.129:56353)
0808 03:52:18.552587 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:18.552602 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:18.552739 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:18.553057 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:18.553062 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:18.553516 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:18.553682 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:18.553692 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:18.554725 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:18.554731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:18.555777 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:18.555794 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:18.556720 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:18.556725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:18.556846 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:18.557363 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:18.557380 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:18.559892 (+ 2512us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":238,"threads_started":1}
W20260808 03:52:18.560243 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:18.552207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:56353 (local address 127.26.146.190:36405)
0808 03:52:18.552434 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:18.552440 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:18.552676 (+ 236us) server_negotiation.cc:407] Connection header received
0808 03:52:18.552864 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:18.552868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:18.552927 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:18.553002 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:18.553845 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:18.554588 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:18.555927 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:18.556571 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:18.556690 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:18.560046 (+ 3356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.129:56353: BlockingRecv error: recv got EOF from 127.26.146.129:56353 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 03:52:19 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:19 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:19.009857 8164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:19.000899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.130:46977)
0808 03:52:19.001458 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:19.001474 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:19.001583 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:19.001859 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:19.001862 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:19.002223 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:19.002378 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:19.002387 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:19.003462 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:19.003466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:19.004608 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:19.004625 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:19.005484 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:19.005490 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:19.005675 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:19.006295 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:19.006317 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:19.009519 (+ 3202us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":270,"threads_started":1}
W20260808 03:52:19.010074 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:19.001119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:46977 (local address 127.26.146.190:36405)
0808 03:52:19.001305 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:19.001312 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:19.001541 (+ 229us) server_negotiation.cc:407] Connection header received
0808 03:52:19.001692 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:19.001696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:19.001759 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:19.001830 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:19.002485 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:19.003325 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:19.004753 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:19.005342 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:19.005438 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:19.009809 (+ 4371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.130:46977: BlockingRecv error: recv got EOF from 127.26.146.130:46977 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 08 03:52:19 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:19 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:19.454998 8167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:19.447181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.131:48669)
0808 03:52:19.447711 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:19.447732 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:19.447886 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:19.448164 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:19.448167 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:19.448605 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:19.448770 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:19.448779 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:19.449690 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:19.449693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:19.450606 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:19.450618 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:19.451227 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:19.451231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:19.451346 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:19.451973 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:19.451990 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:19.454647 (+ 2657us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":242,"threads_started":1}
W20260808 03:52:19.455099 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:19.447257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:48669 (local address 127.26.146.190:36405)
0808 03:52:19.447453 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:19.447459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:19.447824 (+ 365us) server_negotiation.cc:407] Connection header received
0808 03:52:19.448006 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:19.448010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:19.448072 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:19.448138 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:19.448888 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:19.449588 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:19.450717 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:19.451122 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:19.451217 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:19.454870 (+ 3653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.131:48669: BlockingRecv error: recv got EOF from 127.26.146.131:48669 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 08 03:52:19 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:19 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:19.569082 8170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:19.560972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.129:38625)
0808 03:52:19.561507 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:19.561524 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:19.561647 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:19.561910 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:19.561913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:19.562382 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:19.562571 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:19.562586 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:19.563934 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:19.563938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:19.564843 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:19.564856 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:19.565460 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:19.565464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:19.565584 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:19.566145 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:19.566162 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:19.568853 (+ 2691us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":291,"threads_started":1}
W20260808 03:52:19.569265 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:19.561048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:38625 (local address 127.26.146.190:36405)
0808 03:52:19.561231 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:19.561235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:19.561582 (+ 347us) server_negotiation.cc:407] Connection header received
0808 03:52:19.561740 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:19.561743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:19.561801 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:19.561879 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:19.562731 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:19.563789 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:19.564954 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:19.565351 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:19.565438 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:19.569035 (+ 3597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.129:38625: BlockingRecv error: recv got EOF from 127.26.146.129:38625 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 03:52:20 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:20 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:20.019661 8175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:20.010892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.130:40747)
0808 03:52:20.011395 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:20.011410 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:20.011550 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:20.011873 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:20.011877 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:20.012286 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:20.012434 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:20.012443 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:20.013444 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:20.013450 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:20.014673 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:20.014686 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:20.015380 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:20.015385 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:20.015550 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:20.016169 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:20.016187 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:20.019411 (+ 3224us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":277,"threads_started":1}
W20260808 03:52:20.020051 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:20.010982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:40747 (local address 127.26.146.190:36405)
0808 03:52:20.011176 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:20.011183 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:20.011498 (+ 315us) server_negotiation.cc:407] Connection header received
0808 03:52:20.011661 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:20.011666 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:20.011728 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:20.011823 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:20.012568 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:20.013288 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:20.014814 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:20.015234 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:20.015340 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:20.019669 (+ 4329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.130:40747: BlockingRecv error: recv got EOF from 127.26.146.130:40747 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 03:52:20 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:20 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:20.467660 8177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:20.456064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.131:35993)
0808 03:52:20.456591 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:20.456614 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:20.456729 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:20.457059 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:20.457063 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:20.457456 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:20.457677 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:20.457688 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:20.458695 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:20.458699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:20.459667 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:20.459680 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:20.460476 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:20.460482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:20.460651 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:20.464497 (+ 3846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:20.464516 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:20.467400 (+ 2884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":268,"threads_started":1}
W20260808 03:52:20.467957 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:20.456106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:35993 (local address 127.26.146.190:36405)
0808 03:52:20.456279 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:20.456284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:20.456680 (+ 396us) server_negotiation.cc:407] Connection header received
0808 03:52:20.456817 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:20.456822 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:20.456883 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:20.456964 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:20.457796 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:20.458553 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:20.459826 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:20.460313 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:20.460424 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:20.467660 (+ 7236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.131:35993: BlockingRecv error: recv got EOF from 127.26.146.131:35993 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 03:52:20 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:20 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:20.578207 8178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:20.569951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.129:49649)
0808 03:52:20.570597 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:20.570613 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:20.570736 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:20.570992 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:20.570995 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:20.571404 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:20.571639 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:20.571650 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:20.572616 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:20.572620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:20.573604 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:20.573618 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:20.574260 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:20.574264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:20.574409 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:20.575009 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:20.575029 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:20.577821 (+ 2792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":256,"threads_started":1}
W20260808 03:52:20.578272 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:20.570049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:49649 (local address 127.26.146.190:36405)
0808 03:52:20.570220 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:20.570224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:20.570674 (+ 450us) server_negotiation.cc:407] Connection header received
0808 03:52:20.570827 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:20.570830 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:20.570888 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:20.570963 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:20.571763 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:20.572467 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:20.573721 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:20.574129 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:20.574221 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:20.578044 (+ 3823us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.129:49649: BlockingRecv error: recv got EOF from 127.26.146.129:49649 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 03:52:21 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:21 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:21.028950 8179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:21.020573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.130:51237)
0808 03:52:21.021164 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:21.021185 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:21.021328 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:21.021693 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:21.021869 (+ 176us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:21.022393 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:21.022570 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:21.022581 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:21.023607 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:21.023613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:21.024579 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:21.024591 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:21.025275 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:21.025281 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:21.025441 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:21.025955 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:21.025972 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:21.028693 (+ 2721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":285,"threads_started":1}
W20260808 03:52:21.029093 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:21.020675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:51237 (local address 127.26.146.190:36405)
0808 03:52:21.020982 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:21.020991 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:21.021262 (+ 271us) server_negotiation.cc:407] Connection header received
0808 03:52:21.021451 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:21.021457 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:21.021554 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:21.021646 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:21.022711 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:21.023475 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:21.024712 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:21.025131 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:21.025246 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:21.028864 (+ 3618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.130:51237: BlockingRecv error: recv got EOF from 127.26.146.130:51237 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 08 03:52:21 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:21 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:21.476713 8180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:21.468582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.131:42457)
0808 03:52:21.469139 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:21.469159 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:21.469287 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:21.469593 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:21.469599 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:21.470040 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:21.470210 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:21.470220 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:21.471192 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:21.471197 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:21.472163 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:21.472175 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:21.472871 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:21.472877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:21.473036 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:21.473607 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:21.473625 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:21.476347 (+ 2722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":233,"threads_started":1}
W20260808 03:52:21.476766 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:21.468651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:42457 (local address 127.26.146.190:36405)
0808 03:52:21.468818 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:21.468823 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:21.469253 (+ 430us) server_negotiation.cc:407] Connection header received
0808 03:52:21.469401 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:21.469406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:21.469471 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:21.469548 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:21.470351 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:21.471061 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:21.472296 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:21.472729 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:21.472834 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:21.476569 (+ 3735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.131:42457: BlockingRecv error: recv got EOF from 127.26.146.131:42457 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 03:52:21 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:21 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:21.587400 8181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:21.579201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.129:48997)
0808 03:52:21.579742 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:21.579760 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:21.579907 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:21.580229 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:21.580233 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:21.580648 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:21.580816 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:21.580825 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:21.581762 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:21.581766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:21.582785 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:21.582798 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:21.583711 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:21.583718 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:21.583900 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:21.584475 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:21.584493 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:21.587170 (+ 2677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":282,"threads_started":1}
W20260808 03:52:21.587576 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:21.579361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:48997 (local address 127.26.146.190:36405)
0808 03:52:21.579521 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:21.579525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:21.579825 (+ 300us) server_negotiation.cc:407] Connection header received
0808 03:52:21.579999 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:21.580002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:21.580077 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:21.580152 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:21.580930 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:21.581644 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:21.582932 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:21.583509 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:21.583619 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:21.587346 (+ 3727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.129:48997: BlockingRecv error: recv got EOF from 127.26.146.129:48997 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 03:52:22 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:22 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:22.037825 8183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:22.029859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.130:34137)
0808 03:52:22.030359 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:22.030374 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:22.030502 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:22.030802 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:22.030806 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:22.031213 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:22.031371 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:22.031380 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:22.032605 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:22.032616 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:22.033527 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:22.033539 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:22.034276 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:22.034282 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:22.034425 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:22.034976 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:22.034993 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:22.037601 (+ 2608us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":243,"threads_started":1}
W20260808 03:52:22.037966 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:22.029950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:34137 (local address 127.26.146.190:36405)
0808 03:52:22.030157 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:22.030164 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:22.030454 (+ 290us) server_negotiation.cc:407] Connection header received
0808 03:52:22.030612 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:22.030616 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:22.030678 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:22.030759 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:22.031524 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:22.032467 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:22.033669 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:22.034095 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:22.034211 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:22.037774 (+ 3563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.130:34137: BlockingRecv error: recv got EOF from 127.26.146.130:34137 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 03:52:22 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:22 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:22.486101 8184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:22.477681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.131:49585)
0808 03:52:22.478180 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:22.478200 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:22.478319 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:22.478623 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:22.478627 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:22.479019 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:22.479175 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:22.479183 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:22.480636 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:22.480639 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:22.481641 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:22.481656 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:22.482378 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:22.482383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:22.482496 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:22.483057 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:22.483075 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:22.485877 (+ 2802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":245,"threads_started":1}
W20260808 03:52:22.486291 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:22.477768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:49585 (local address 127.26.146.190:36405)
0808 03:52:22.478002 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:22.478008 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:22.478307 (+ 299us) server_negotiation.cc:407] Connection header received
0808 03:52:22.478408 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:22.478414 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:22.478489 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:22.478587 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:22.479308 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:22.480496 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:22.481781 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:22.482220 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:22.482322 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:22.486051 (+ 3729us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.131:49585: BlockingRecv error: recv got EOF from 127.26.146.131:49585 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 03:52:22 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:22 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:22.596554 8185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:22.588346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.129:44481)
0808 03:52:22.588867 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:22.588887 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:22.589036 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:22.589437 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:22.589442 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:22.589989 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:22.590160 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:22.590171 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:22.591204 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:22.591210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:22.592177 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:22.592189 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:22.592845 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:22.592850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:22.593021 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:22.593524 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:22.593540 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:22.596237 (+ 2697us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":247,"threads_started":1}
W20260808 03:52:22.596669 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:22.588403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:44481 (local address 127.26.146.190:36405)
0808 03:52:22.588597 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:22.588604 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:22.588990 (+ 386us) server_negotiation.cc:407] Connection header received
0808 03:52:22.589168 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:22.589172 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:22.589293 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:22.589377 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:22.590305 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:22.591076 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:22.592315 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:22.592701 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:22.592838 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:22.596447 (+ 3609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.129:44481: BlockingRecv error: recv got EOF from 127.26.146.129:44481 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 03:52:23 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:23 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:23.046873 8186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:23.038715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.130:59281)
0808 03:52:23.039250 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:23.039266 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:23.039377 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:23.039631 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:23.039634 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:23.040047 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:23.040227 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:23.040236 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:23.041224 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:23.041230 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:23.042182 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:23.042195 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:23.042898 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:23.042903 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:23.043061 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:23.043661 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:23.043679 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:23.046636 (+ 2957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":279,"threads_started":1}
W20260808 03:52:23.046985 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:23.038785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:59281 (local address 127.26.146.190:36405)
0808 03:52:23.038953 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:23.038958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:23.039328 (+ 370us) server_negotiation.cc:407] Connection header received
0808 03:52:23.039462 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:23.039465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:23.039524 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:23.039603 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:23.040331 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:23.041071 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:23.042325 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:23.042743 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:23.042849 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:23.046794 (+ 3945us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.130:59281: BlockingRecv error: recv got EOF from 127.26.146.130:59281 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 03:52:23 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:23 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:23.495055 8187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:23.487055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.131:51873)
0808 03:52:23.487574 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:23.487596 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:23.487722 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:23.488086 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:23.488090 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:23.488498 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:23.488664 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:23.488673 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:23.489639 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:23.489643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:23.490585 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:23.490598 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:23.491285 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:23.491291 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:23.491456 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:23.492018 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:23.492036 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:23.494741 (+ 2705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":293,"threads_started":1}
W20260808 03:52:23.495172 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:23.487171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:51873 (local address 127.26.146.190:36405)
0808 03:52:23.487344 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:23.487351 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:23.487679 (+ 328us) server_negotiation.cc:407] Connection header received
0808 03:52:23.487858 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:23.487863 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:23.487945 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:23.488026 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:23.488810 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:23.489505 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:23.490735 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:23.491138 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:23.491243 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:23.494948 (+ 3705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.131:51873: BlockingRecv error: recv got EOF from 127.26.146.131:51873 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 03:52:23 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:23 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:23.605973 8188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:23.597499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.129:46695)
0808 03:52:23.597989 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:23.598005 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:23.598139 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:23.598435 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:23.598438 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:23.598829 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:23.598997 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:23.599014 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:23.600003 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:23.600007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:23.600947 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:23.600958 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:23.601686 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:23.601691 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:23.601814 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:23.602442 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:23.602461 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:23.605667 (+ 3206us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":277,"threads_started":1}
W20260808 03:52:23.606115 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:23.597596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:46695 (local address 127.26.146.190:36405)
0808 03:52:23.597762 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:23.597767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:23.598069 (+ 302us) server_negotiation.cc:407] Connection header received
0808 03:52:23.598232 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:23.598235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:23.598326 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:23.598403 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:23.599149 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:23.599905 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:23.601051 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:23.601466 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:23.601552 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:23.605873 (+ 4321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.129:46695: BlockingRecv error: recv got EOF from 127.26.146.129:46695 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 03:52:24 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:24 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:24.055258 8190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:24.047790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.130:51849)
0808 03:52:24.048302 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:24.048320 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:24.048431 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:24.048684 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:24.048688 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:24.049071 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:24.049217 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:24.049225 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:24.050104 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:24.050108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:24.051021 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:24.051032 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:24.051664 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:24.051669 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:24.051782 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:24.052331 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:24.052349 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:24.055017 (+ 2668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":293,"threads_started":1}
W20260808 03:52:24.055420 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:24.047849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:51849 (local address 127.26.146.190:36405)
0808 03:52:24.048002 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:24.048007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:24.048384 (+ 377us) server_negotiation.cc:407] Connection header received
0808 03:52:24.048514 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:24.048518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:24.048577 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:24.048653 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:24.049327 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:24.050001 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:24.051123 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:24.051543 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:24.051628 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:24.055212 (+ 3584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.130:51849: BlockingRecv error: recv got EOF from 127.26.146.130:51849 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 03:52:24 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:24 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:24.504133 8191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:24.496043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.131:58491)
0808 03:52:24.496543 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:24.496565 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:24.496714 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:24.497040 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:24.497045 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:24.497458 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:24.497610 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:24.497619 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:24.498656 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:24.498662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:24.499837 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:24.499852 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:24.500637 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:24.500642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:24.500771 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:24.501296 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:24.501312 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:24.503908 (+ 2596us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":270,"threads_started":1}
W20260808 03:52:24.504362 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:24.496246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:58491 (local address 127.26.146.190:36405)
0808 03:52:24.496456 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:24.496467 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:24.496637 (+ 170us) server_negotiation.cc:407] Connection header received
0808 03:52:24.496828 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:24.496833 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:24.496906 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:24.496988 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:24.497766 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:24.498503 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:24.499971 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:24.500507 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:24.500603 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:24.504088 (+ 3485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.131:58491: BlockingRecv error: recv got EOF from 127.26.146.131:58491 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 08 03:52:24 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:24 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:24.614650 8192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:24.606956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.129:38009)
0808 03:52:24.607436 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:24.607452 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:24.607566 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:24.607870 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:24.607873 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:24.608281 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:24.608456 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:24.608464 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:24.609423 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:24.609426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:24.610310 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:24.610320 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:24.610928 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:24.610932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:24.611052 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:24.611634 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:24.611661 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:24.614405 (+ 2744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":249,"threads_started":1}
W20260808 03:52:24.614940 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:24.607057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:38009 (local address 127.26.146.190:36405)
0808 03:52:24.607244 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:24.607250 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:24.607520 (+ 270us) server_negotiation.cc:407] Connection header received
0808 03:52:24.607656 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:24.607659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:24.607740 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:24.607852 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:24.608583 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:24.609325 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:24.610412 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:24.610818 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:24.610905 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:24.614675 (+ 3770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.129:38009: BlockingRecv error: recv got EOF from 127.26.146.129:38009 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 08 03:52:25 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:25 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:25.064615 8193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:25.056185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.130:39637)
0808 03:52:25.056671 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:25.056686 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:25.056816 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:25.057125 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:25.057130 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:25.057567 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:25.057741 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:25.057751 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:25.058764 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:25.058768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:25.059657 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:25.059669 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:25.060406 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:25.060413 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:25.060597 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:25.061174 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:25.061191 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:25.064370 (+ 3179us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":255,"threads_started":1}
W20260808 03:52:25.064792 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:25.056270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:39637 (local address 127.26.146.190:36405)
0808 03:52:25.056463 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:25.056470 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:25.056762 (+ 292us) server_negotiation.cc:407] Connection header received
0808 03:52:25.056924 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:25.056930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:25.056993 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:25.057071 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:25.057879 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:25.058621 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:25.059795 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:25.060255 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:25.060370 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:25.064568 (+ 4198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.130:39637: BlockingRecv error: recv got EOF from 127.26.146.130:39637 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 03:52:25 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:25 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:25.516925 8194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:25.505127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.131:57533)
0808 03:52:25.505641 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:25.505662 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:25.505794 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:25.506123 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:25.506128 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:25.506636 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:25.506823 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:25.506833 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:25.507907 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:25.507914 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:25.508796 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:25.508808 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:25.509479 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:25.509485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:25.509647 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:25.513798 (+ 4151us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:25.513815 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:25.516676 (+ 2861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":261,"threads_started":1}
W20260808 03:52:25.517123 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:25.505200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:57533 (local address 127.26.146.190:36405)
0808 03:52:25.505456 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:25.505462 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:25.505748 (+ 286us) server_negotiation.cc:407] Connection header received
0808 03:52:25.505914 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:25.505921 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:25.505992 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:25.506071 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:25.506991 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:25.507772 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:25.508936 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:25.509339 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:25.509444 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:25.516886 (+ 7442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.131:57533: BlockingRecv error: recv got EOF from 127.26.146.131:57533 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 08 03:52:25 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:25 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:25.623397 8195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:25.615645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.129:51663)
0808 03:52:25.616165 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:25.616181 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:25.616312 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:25.616584 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:25.616587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:25.616961 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:25.617121 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:25.617129 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:25.618090 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:25.618093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:25.619000 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:25.619010 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:25.619653 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:25.619657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:25.619779 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:25.620395 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:25.620414 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:25.623125 (+ 2711us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":299,"threads_started":1}
W20260808 03:52:25.623598 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:25.615728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:51663 (local address 127.26.146.190:36405)
0808 03:52:25.615910 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:25.615915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:25.616258 (+ 343us) server_negotiation.cc:407] Connection header received
0808 03:52:25.616402 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:25.616406 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:25.616467 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:25.616545 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:25.617228 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:25.617979 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:25.619101 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:25.619549 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:25.619639 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:25.623354 (+ 3715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.129:51663: BlockingRecv error: recv got EOF from 127.26.146.129:51663 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 03:52:26 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:26 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:26.073673 8196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:26.065463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.130:40957)
0808 03:52:26.065951 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:26.065967 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:26.066079 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:26.066361 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:26.066364 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:26.066795 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:26.066961 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:26.066974 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:26.068364 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:26.068369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:26.069329 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:26.069343 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:26.069978 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:26.069982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:26.070102 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:26.070626 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:26.070642 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:26.073432 (+ 2790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":288,"threads_started":1}
W20260808 03:52:26.073874 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:26.065508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:40957 (local address 127.26.146.190:36405)
0808 03:52:26.065719 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:26.065726 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:26.066037 (+ 311us) server_negotiation.cc:407] Connection header received
0808 03:52:26.066172 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:26.066175 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:26.066239 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:26.066315 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:26.067101 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:26.068202 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:26.069455 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:26.069873 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:26.069966 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:26.073640 (+ 3674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.130:40957: BlockingRecv error: recv got EOF from 127.26.146.130:40957 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 08 03:52:26 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:26 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:26.526420 8197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:26.517784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.131:58563)
0808 03:52:26.518278 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:26.518298 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:26.518425 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:26.518746 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:26.518750 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:26.519140 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:26.519295 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:26.519304 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:26.520509 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:26.520513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:26.521634 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:26.521648 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:26.522354 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:26.522360 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:26.522532 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:26.523233 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:26.523256 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:26.526176 (+ 2920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":263,"threads_started":1}
W20260808 03:52:26.526573 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:26.517848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:58563 (local address 127.26.146.190:36405)
0808 03:52:26.518027 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:26.518034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:26.518380 (+ 346us) server_negotiation.cc:407] Connection header received
0808 03:52:26.518532 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:26.518538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:26.518602 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:26.518682 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:26.519430 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:26.520312 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:26.521773 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:26.522215 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:26.522304 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:26.526368 (+ 4064us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.131:58563: BlockingRecv error: recv got EOF from 127.26.146.131:58563 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 03:52:26 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:26 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:26.632726 8198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:26.624309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.129:44287)
0808 03:52:26.624831 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:26.624846 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:26.624962 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:26.625277 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:26.625282 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:26.625693 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:26.625874 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:26.625884 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:26.626854 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:26.626858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:26.628188 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:26.628204 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:26.629048 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:26.629054 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:26.629171 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:26.629694 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:26.629718 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:26.632486 (+ 2768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":252,"threads_started":1}
W20260808 03:52:26.632871 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:26.624400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:44287 (local address 127.26.146.190:36405)
0808 03:52:26.624558 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:26.624563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:26.624911 (+ 348us) server_negotiation.cc:407] Connection header received
0808 03:52:26.625052 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:26.625056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:26.625114 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:26.625194 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:26.625987 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:26.626721 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:26.628341 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:26.628879 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:26.628971 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:26.632670 (+ 3699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.129:44287: BlockingRecv error: recv got EOF from 127.26.146.129:44287 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 03:52:27 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:27 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:27.082775 8199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:27.074441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.130:40043)
0808 03:52:27.074970 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:27.074986 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:27.075102 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:27.075390 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:27.075394 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:27.075828 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:27.075995 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:27.076004 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:27.077222 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:27.077226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:27.078305 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:27.078318 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:27.079046 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:27.079052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:27.079204 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:27.079748 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:27.079765 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:27.082444 (+ 2679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":289,"threads_started":1}
W20260808 03:52:27.082913 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:27.074618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:40043 (local address 127.26.146.190:36405)
0808 03:52:27.074817 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:27.074823 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:27.075058 (+ 235us) server_negotiation.cc:407] Connection header received
0808 03:52:27.075208 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:27.075212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:27.075274 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:27.075350 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:27.076128 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:27.077060 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:27.078446 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:27.078859 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:27.078965 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:27.082683 (+ 3718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.130:40043: BlockingRecv error: recv got EOF from 127.26.146.130:40043 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 03:52:27 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:27 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:27.535159 8201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:27.527289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.131:32895)
0808 03:52:27.527783 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:27.527829 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:27.527953 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:27.528210 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:27.528213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:27.528600 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:27.528764 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:27.528772 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:27.529744 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:27.529750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:27.530747 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:27.530760 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:27.531450 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:27.531455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:27.531612 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:27.532198 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:27.532216 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:27.534900 (+ 2684us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":247,"threads_started":1}
W20260808 03:52:27.535290 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:27.527352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:32895 (local address 127.26.146.190:36405)
0808 03:52:27.527551 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:27.527559 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:27.527894 (+ 335us) server_negotiation.cc:407] Connection header received
0808 03:52:27.528036 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:27.528040 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:27.528104 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:27.528180 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:27.528889 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:27.529593 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:27.530883 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:27.531294 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:27.531384 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:27.535099 (+ 3715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.131:32895: BlockingRecv error: recv got EOF from 127.26.146.131:32895 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 03:52:27 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:27 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:27.641357 8206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:27.633567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.129:53807)
0808 03:52:27.634030 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:27.634047 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:27.634163 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:27.634427 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:27.634430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:27.634800 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:27.634983 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:27.634992 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:27.636009 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:27.636013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:27.636966 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:27.636979 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:27.637575 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:27.637579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:27.637691 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:27.638244 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:27.638261 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:27.641021 (+ 2760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":236,"threads_started":1}
W20260808 03:52:27.641391 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:27.633627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:53807 (local address 127.26.146.190:36405)
0808 03:52:27.633922 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:27.633926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:27.634110 (+ 184us) server_negotiation.cc:407] Connection header received
0808 03:52:27.634252 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:27.634256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:27.634315 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:27.634393 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:27.635090 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:27.635903 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:27.637071 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:27.637470 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:27.637556 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:27.641199 (+ 3643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.129:53807: BlockingRecv error: recv got EOF from 127.26.146.129:53807 (error 108)
Metrics: {"server-negotiator.queue_time_us":171}
Aug 08 03:52:28 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:28 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:28.091650 8207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:28.083722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.130:51417)
0808 03:52:28.084200 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:28.084217 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:28.084338 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:28.084616 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:28.084619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:28.084987 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:28.085200 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:28.085215 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:28.086141 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:28.086144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:28.087029 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:28.087038 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:28.087643 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:28.087648 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:28.087761 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:28.088354 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:28.088371 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:28.091242 (+ 2871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":261,"threads_started":1}
W20260808 03:52:28.091711 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:28.083825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:51417 (local address 127.26.146.190:36405)
0808 03:52:28.084026 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:28.084033 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:28.084294 (+ 261us) server_negotiation.cc:407] Connection header received
0808 03:52:28.084432 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:28.084436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:28.084498 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:28.084573 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:28.085319 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:28.086021 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:28.087135 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:28.087533 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:28.087619 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:28.091441 (+ 3822us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.130:51417: BlockingRecv error: recv got EOF from 127.26.146.130:51417 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 03:52:28 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:28 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.131@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:28.544096 8208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:28.536159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.131:42059)
0808 03:52:28.536692 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:28.536718 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:28.536848 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:28.537180 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:28.537186 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:28.537579 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:28.537783 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:28.537793 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:28.538820 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:28.538826 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:28.539731 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:28.539743 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:28.540474 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:28.540479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:28.540652 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:28.541201 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:28.541218 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:28.543854 (+ 2636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":280,"threads_started":1}
W20260808 03:52:28.544261 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:28.536249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:42059 (local address 127.26.146.190:36405)
0808 03:52:28.536448 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:28.536455 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:28.536807 (+ 352us) server_negotiation.cc:407] Connection header received
0808 03:52:28.536976 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:28.536981 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:28.537044 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:28.537133 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:28.537933 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:28.538666 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:28.539884 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:28.540334 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:28.540444 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:28.544056 (+ 3612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.131:42059: BlockingRecv error: recv got EOF from 127.26.146.131:42059 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 03:52:28 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:28 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.129@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:28.650768 8209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:28.642413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.129:58569)
0808 03:52:28.642955 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:28.642970 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:28.643096 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:28.643410 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:28.643415 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:28.643896 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:28.644072 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:28.644083 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:28.645134 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:28.645138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:28.646044 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:28.646057 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:28.646780 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:28.646786 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:28.646935 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:28.647525 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:28.647545 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:28.650396 (+ 2851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":333,"threads_started":1}
W20260808 03:52:28.650882 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:28.642481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:58569 (local address 127.26.146.190:36405)
0808 03:52:28.642680 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:28.642686 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:28.643051 (+ 365us) server_negotiation.cc:407] Connection header received
0808 03:52:28.643213 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:28.643219 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:28.643281 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:28.643360 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:28.644234 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:28.644982 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:28.646186 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:28.646637 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:28.646744 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:28.650678 (+ 3934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.129:58569: BlockingRecv error: recv got EOF from 127.26.146.129:58569 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 03:52:29 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 03:52:29 dist-test-slave-jlzn krb5kdc[6715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.146.130@KRBTEST.COM for oryx/127.26.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 03:52:29.101123 8211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:29.092842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36405 (local address 127.26.146.130:47367)
0808 03:52:29.093326 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:29.093342 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:29.093470 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:29.093777 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:29.093781 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:29.094205 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:29.094373 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:29.094381 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:29.095384 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:29.095388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:29.096489 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:29.096504 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:29.097223 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:29.097229 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:29.097391 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:29.098005 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:29.098024 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:29.100781 (+ 2757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.146.190:36405: Server oryx/127.26.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":262,"threads_started":1}
W20260808 03:52:29.101254 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:29.092950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:47367 (local address 127.26.146.190:36405)
0808 03:52:29.093165 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:29.093171 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:29.093420 (+ 249us) server_negotiation.cc:407] Connection header received
0808 03:52:29.093577 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:29.093583 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:29.093645 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:29.093723 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:29.094514 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:29.095236 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:29.096631 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:29.097080 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:29.097189 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:29.100997 (+ 3808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.146.130:47367: BlockingRecv error: recv got EOF from 127.26.146.130:47367 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 03:52:29.384760 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 7560
I20260808 03:52:29.396481 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:29.385845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50812 (local address 127.26.146.129:36011)
0808 03:52:29.386318 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:29.386326 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:29.386353 (+ 27us) server_negotiation.cc:407] Connection header received
0808 03:52:29.386412 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:29.386419 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:29.386660 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:29.386814 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:29.388265 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:29.389282 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:29.390391 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:29.390944 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:29.391045 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:29.393456 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:29.393465 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:29.393479 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:29.393548 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:29.395993 (+ 2445us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.26.146.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 03:52:29.396254 (+ 261us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50812: Request ticket server kudu/127.26.146.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":200,"threads_started":1}
W20260808 03:52:29.396603 8214 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50812: Request ticket server kudu/127.26.146.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 03:52:29.396674 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 7560: Not authorized: Client connection negotiation failed: client connection to 127.26.146.129:36011: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.26.146.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 03:52:29.396908 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 7560
I20260808 03:52:29.427232 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 7765
I20260808 03:52:29.439136 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:29.428451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52730 (local address 127.26.146.130:34471)
0808 03:52:29.429103 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:29.429111 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:29.429142 (+ 31us) server_negotiation.cc:407] Connection header received
0808 03:52:29.429204 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:29.429210 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:29.429422 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:29.429571 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:29.430803 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:29.431968 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:29.433131 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:29.433729 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:29.433831 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:29.436173 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:29.436182 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:29.436196 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:29.436232 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:29.438601 (+ 2369us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.26.146.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 03:52:29.438860 (+ 259us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52730: Request ticket server kudu/127.26.146.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":337,"threads_started":1}
W20260808 03:52:29.439247 8215 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52730: Request ticket server kudu/127.26.146.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 03:52:29.439250 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 7765: Not authorized: Client connection negotiation failed: client connection to 127.26.146.130:34471: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.26.146.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 03:52:29.439406 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 7765
I20260808 03:52:29.463711 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 7959
I20260808 03:52:29.475512 8218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:29.464670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38682 (local address 127.26.146.131:45815)
0808 03:52:29.465259 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:29.465266 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:29.465293 (+ 27us) server_negotiation.cc:407] Connection header received
0808 03:52:29.465356 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:29.465363 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:29.465584 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:29.465744 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:29.467038 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:29.468254 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:29.469483 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:29.470028 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:29.470144 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:29.472628 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:29.472635 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:29.472649 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:29.472679 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:29.475013 (+ 2334us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.26.146.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 03:52:29.475260 (+ 247us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38682: Request ticket server kudu/127.26.146.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":280,"threads_started":1}
W20260808 03:52:29.475621 8218 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38682: Request ticket server kudu/127.26.146.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 03:52:29.475677 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 7959: Not authorized: Client connection negotiation failed: client connection to 127.26.146.131:45815: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.26.146.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 03:52:29.475879 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 7959
I20260808 03:52:29.499756 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 7459
I20260808 03:52:29.501009 7548 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:29.618618 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 7459
2026-08-08T03:52:29Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21211 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T03:52:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:52:29Z Disabled control of system clock
I20260808 03:52:29.707511 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:37853
--webserver_interface=127.26.146.190
--webserver_port=0
--builtin_ntp_servers=127.26.146.148:41515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:37853
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 03:52:30.031006 8225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:30.031289 8225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:30.031404 8225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:30.038465 8225 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:52:30.038565 8225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:30.038630 8225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:30.038707 8225 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:52:30.038766 8225 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:52:30.047653 8225 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:41515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:37853
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:37853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:30.048949 8225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:30.050731 8225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:30.062438 8233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:30.062403 8231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:30.062853 8225 server_base.cc:1034] running on GCE node
W20260808 03:52:30.062892 8230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:30.063758 8225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:30.065121 8225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:30.066334 8225 hybrid_clock.cc:648] HybridClock initialized: now 1786161150066285 us; error 32 us; skew 500 ppm
I20260808 03:52:30.066655 8225 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:52:30.068768 8225 webserver.cc:460] Webserver started at http://127.26.146.190:35925/ using document root <none> and password file <none>
I20260808 03:52:30.069414 8225 fs_manager.cc:359] Metadata directory not provided
I20260808 03:52:30.069516 8225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:52:30.069855 8225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:52:30.072741 8225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "3d0cbdac0e8842749302473799cdbe79"
format_stamp: "Formatted at 2026-08-08 03:52:30 on dist-test-slave-jlzn"
server_key: "d4676c8628404567cd7531b73275215f"
server_key_iv: "3cb12df03d9ff25738ea7729cc6c7c22"
server_key_version: "encryptionkey@0"
I20260808 03:52:30.073482 8225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "3d0cbdac0e8842749302473799cdbe79"
format_stamp: "Formatted at 2026-08-08 03:52:30 on dist-test-slave-jlzn"
server_key: "d4676c8628404567cd7531b73275215f"
server_key_iv: "3cb12df03d9ff25738ea7729cc6c7c22"
server_key_version: "encryptionkey@0"
I20260808 03:52:30.078359 8225 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:52:30.081648 8244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:30.082878 8225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:52:30.083057 8225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "3d0cbdac0e8842749302473799cdbe79"
format_stamp: "Formatted at 2026-08-08 03:52:30 on dist-test-slave-jlzn"
server_key: "d4676c8628404567cd7531b73275215f"
server_key_iv: "3cb12df03d9ff25738ea7729cc6c7c22"
server_key_version: "encryptionkey@0"
I20260808 03:52:30.083252 8225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:52:30.097481 8225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:52:30.098228 8225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:52:30.098486 8225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:52:30.115773 8225 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:37853
I20260808 03:52:30.115779 8316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:37853 every 8 connection(s)
I20260808 03:52:30.117116 8225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:52:30.117771 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 8225
I20260808 03:52:30.117942 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:52:30.118753 27210 external_mini_cluster.cc:1428] Setting key fe4d46ac026a6f4de75f1b9d185f0b75
I20260808 03:52:30.121089 8317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:52:30.134891 8317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d0cbdac0e8842749302473799cdbe79: Bootstrap starting.
I20260808 03:52:30.136965 8322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:30.120836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43228 (local address 127.26.146.190:37853)
0808 03:52:30.124615 (+ 3779us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:30.124630 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:30.124746 (+ 116us) server_negotiation.cc:407] Connection header received
0808 03:52:30.127105 (+ 2359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:30.127133 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:30.127707 (+ 574us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:30.128178 (+ 471us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:30.128765 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:30.130580 (+ 1815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:30.131908 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:30.132954 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:30.133131 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:30.134302 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:30.134312 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:30.134328 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:52:30.134992 (+ 664us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:30.135100 (+ 108us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:52:30.135125 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:30.135241 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:30.135424 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 03:52:30.135713 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3258,"thread_start_us":3013,"threads_started":1}
I20260808 03:52:30.138549 8317 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d0cbdac0e8842749302473799cdbe79: Neither blocks nor log segments found. Creating new log.
I20260808 03:52:30.139853 8317 log.cc:824] T 00000000000000000000000000000000 P 3d0cbdac0e8842749302473799cdbe79: Log is configured to *not* fsync() on all Append() calls
I20260808 03:52:30.142442 8317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d0cbdac0e8842749302473799cdbe79: No bootstrap required, opened a new log
I20260808 03:52:30.149498 8317 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d0cbdac0e8842749302473799cdbe79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0cbdac0e8842749302473799cdbe79" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37853 } }
I20260808 03:52:30.149920 8317 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d0cbdac0e8842749302473799cdbe79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:52:30.150041 8317 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d0cbdac0e8842749302473799cdbe79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0cbdac0e8842749302473799cdbe79, State: Initialized, Role: FOLLOWER
I20260808 03:52:30.150629 8317 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d0cbdac0e8842749302473799cdbe79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0cbdac0e8842749302473799cdbe79" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37853 } }
I20260808 03:52:30.150837 8317 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d0cbdac0e8842749302473799cdbe79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:52:30.150938 8317 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d0cbdac0e8842749302473799cdbe79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:52:30.151057 8317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d0cbdac0e8842749302473799cdbe79 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:52:30.153164 8317 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d0cbdac0e8842749302473799cdbe79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0cbdac0e8842749302473799cdbe79" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37853 } }
I20260808 03:52:30.153587 8317 leader_election.cc:302] T 00000000000000000000000000000000 P 3d0cbdac0e8842749302473799cdbe79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d0cbdac0e8842749302473799cdbe79; no voters:
I20260808 03:52:30.154116 8317 leader_election.cc:288] T 00000000000000000000000000000000 P 3d0cbdac0e8842749302473799cdbe79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:52:30.154242 8325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d0cbdac0e8842749302473799cdbe79 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:52:30.154909 8325 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d0cbdac0e8842749302473799cdbe79 [term 1 LEADER]: Becoming Leader. State: Replica: 3d0cbdac0e8842749302473799cdbe79, State: Running, Role: LEADER
I20260808 03:52:30.155485 8325 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d0cbdac0e8842749302473799cdbe79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0cbdac0e8842749302473799cdbe79" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37853 } }
I20260808 03:52:30.156036 8317 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d0cbdac0e8842749302473799cdbe79 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:52:30.158332 8329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d0cbdac0e8842749302473799cdbe79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d0cbdac0e8842749302473799cdbe79. Latest consensus state: current_term: 1 leader_uuid: "3d0cbdac0e8842749302473799cdbe79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0cbdac0e8842749302473799cdbe79" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37853 } } }
I20260808 03:52:30.158717 8329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d0cbdac0e8842749302473799cdbe79 [sys.catalog]: This master's current role is: LEADER
I20260808 03:52:30.159528 8328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d0cbdac0e8842749302473799cdbe79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d0cbdac0e8842749302473799cdbe79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0cbdac0e8842749302473799cdbe79" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 37853 } } }
I20260808 03:52:30.159940 8328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d0cbdac0e8842749302473799cdbe79 [sys.catalog]: This master's current role is: LEADER
I20260808 03:52:30.160570 8341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:52:30.165652 8341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:52:30.173031 8341 catalog_manager.cc:1347] Generated new cluster ID: f6fb8f14676e4783b7e8f50e8449f8c3
I20260808 03:52:30.173163 8341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:52:30.181479 8341 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:52:30.182634 8341 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:52:30.195652 8341 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d0cbdac0e8842749302473799cdbe79: Generated new TSK 0
I20260808 03:52:30.196453 8341 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:52:30.206449 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:37853
--builtin_ntp_servers=127.26.146.148:41515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 03:52:30.538826 8357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:30.539103 8357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:30.539186 8357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:30.546264 8357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:30.546356 8357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:30.546454 8357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:52:30.555706 8357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:41515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:37853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:30.557178 8357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:30.558941 8357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:30.571874 8367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:30.571877 8366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:30.572335 8370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:30.572661 8357 server_base.cc:1034] running on GCE node
I20260808 03:52:30.573263 8357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:30.574167 8357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:30.575389 8357 hybrid_clock.cc:648] HybridClock initialized: now 1786161150575349 us; error 28 us; skew 500 ppm
I20260808 03:52:30.575696 8357 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:52:30.580554 8357 webserver.cc:460] Webserver started at http://127.26.146.129:36239/ using document root <none> and password file <none>
I20260808 03:52:30.581252 8357 fs_manager.cc:359] Metadata directory not provided
I20260808 03:52:30.581355 8357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:52:30.581668 8357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:52:30.584527 8357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "06d8befa6e8b44fab5def39d244be221"
format_stamp: "Formatted at 2026-08-08 03:52:30 on dist-test-slave-jlzn"
server_key: "4c1aaf9c08408253206e18ed5b877fef"
server_key_iv: "3e313befd54f90af332b63755d3c28b6"
server_key_version: "encryptionkey@0"
I20260808 03:52:30.585291 8357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "06d8befa6e8b44fab5def39d244be221"
format_stamp: "Formatted at 2026-08-08 03:52:30 on dist-test-slave-jlzn"
server_key: "4c1aaf9c08408253206e18ed5b877fef"
server_key_iv: "3e313befd54f90af332b63755d3c28b6"
server_key_version: "encryptionkey@0"
I20260808 03:52:30.590219 8357 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:52:30.593513 8379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:30.594705 8357 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 03:52:30.594861 8357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "06d8befa6e8b44fab5def39d244be221"
format_stamp: "Formatted at 2026-08-08 03:52:30 on dist-test-slave-jlzn"
server_key: "4c1aaf9c08408253206e18ed5b877fef"
server_key_iv: "3e313befd54f90af332b63755d3c28b6"
server_key_version: "encryptionkey@0"
I20260808 03:52:30.595052 8357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:52:30.632030 8357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:52:30.632869 8357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:52:30.633158 8357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:52:30.634105 8357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:52:30.635527 8357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:52:30.635644 8357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:30.635751 8357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:52:30.635852 8357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:30.665953 8357 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:38121
I20260808 03:52:30.666007 8532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:38121 every 8 connection(s)
I20260808 03:52:30.667191 8357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:52:30.668286 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 8357
I20260808 03:52:30.668469 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:52:30.669335 27210 external_mini_cluster.cc:1428] Setting key 663085b6226aa8790a4432c771ad55c5
I20260808 03:52:30.672376 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:37853
--builtin_ntp_servers=127.26.146.148:41515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 03:52:30.680315 8535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:30.669941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:36817 (local address 127.26.146.190:37853)
0808 03:52:30.670735 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:30.670741 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:30.671828 (+ 1087us) server_negotiation.cc:407] Connection header received
0808 03:52:30.673635 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:30.673641 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:30.673728 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:30.673842 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:30.674843 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:30.675782 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:30.677297 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:30.677832 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:30.677947 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:30.679478 (+ 1531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:30.679484 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:30.679488 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:52:30.679509 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:30.679554 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:52:30.679568 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:30.679650 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:30.679902 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 03:52:30.679998 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":568,"threads_started":1}
I20260808 03:52:30.681212 8536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:30.670614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37853 (local address 127.26.146.129:36817)
0808 03:52:30.671685 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:30.671722 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:30.673366 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:30.674015 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:30.674022 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:30.674123 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:30.674677 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:30.674694 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:30.675931 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:30.675935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:30.677150 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:30.677163 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:30.678150 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:30.678160 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:30.678473 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:30.679160 (+ 687us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:52:30.679162 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:30.679211 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:52:30.679291 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:52:30.679667 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:52:30.679673 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:52:30.679679 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 03:52:30.679878 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 03:52:30.680173 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":303,"threads_started":1}
I20260808 03:52:30.682685 8533 heartbeater.cc:347] Connected to a master server at 127.26.146.190:37853
I20260808 03:52:30.682971 8533 heartbeater.cc:464] Registering TS with master...
I20260808 03:52:30.683727 8533 heartbeater.cc:511] Master 127.26.146.190:37853 requested a full tablet report, sending...
I20260808 03:52:30.685936 8267 ts_manager.cc:194] Registered new tserver with Master: 06d8befa6e8b44fab5def39d244be221 (127.26.146.129:38121)
I20260808 03:52:30.687918 8267 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.146.129:36817
W20260808 03:52:31.003281 8537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:31.003551 8537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:31.003635 8537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:31.010615 8537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:31.010711 8537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:31.010820 8537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:52:31.019498 8537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:41515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:37853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:31.020752 8537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:31.022493 8537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:31.035003 8542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:31.035068 8543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:31.035202 8546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:31.035770 8537 server_base.cc:1034] running on GCE node
I20260808 03:52:31.036401 8537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:31.037285 8537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:31.038501 8537 hybrid_clock.cc:648] HybridClock initialized: now 1786161151038452 us; error 36 us; skew 500 ppm
I20260808 03:52:31.038812 8537 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:52:31.040992 8537 webserver.cc:460] Webserver started at http://127.26.146.130:36631/ using document root <none> and password file <none>
I20260808 03:52:31.041654 8537 fs_manager.cc:359] Metadata directory not provided
I20260808 03:52:31.041769 8537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:52:31.042080 8537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:52:31.044876 8537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "6c36bb55593742ef8aea3b34feccf9d7"
format_stamp: "Formatted at 2026-08-08 03:52:31 on dist-test-slave-jlzn"
server_key: "57fb34fd289cdfeb5112c7d01804984f"
server_key_iv: "f4a2759b454cf0187b3f9c433cea2965"
server_key_version: "encryptionkey@0"
I20260808 03:52:31.045643 8537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "6c36bb55593742ef8aea3b34feccf9d7"
format_stamp: "Formatted at 2026-08-08 03:52:31 on dist-test-slave-jlzn"
server_key: "57fb34fd289cdfeb5112c7d01804984f"
server_key_iv: "f4a2759b454cf0187b3f9c433cea2965"
server_key_version: "encryptionkey@0"
I20260808 03:52:31.050786 8537 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:52:31.054241 8557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:31.055696 8537 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:52:31.055877 8537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "6c36bb55593742ef8aea3b34feccf9d7"
format_stamp: "Formatted at 2026-08-08 03:52:31 on dist-test-slave-jlzn"
server_key: "57fb34fd289cdfeb5112c7d01804984f"
server_key_iv: "f4a2759b454cf0187b3f9c433cea2965"
server_key_version: "encryptionkey@0"
I20260808 03:52:31.056124 8537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:52:31.081128 8537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:52:31.081923 8537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:52:31.082208 8537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:52:31.083127 8537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:52:31.084563 8537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:52:31.084652 8537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:31.084759 8537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:52:31.084836 8537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:31.115248 8537 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:42813
I20260808 03:52:31.115271 8710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:42813 every 8 connection(s)
I20260808 03:52:31.116887 8537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:52:31.121212 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 8537
I20260808 03:52:31.121377 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:52:31.121933 27210 external_mini_cluster.cc:1428] Setting key 7dd11ed702b6f5c17b38edfa322eb265
I20260808 03:52:31.125154 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:37853
--builtin_ntp_servers=127.26.146.148:41515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 03:52:31.131903 8535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:31.119705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:41631 (local address 127.26.146.190:37853)
0808 03:52:31.119909 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:31.119914 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:31.121604 (+ 1690us) server_negotiation.cc:407] Connection header received
0808 03:52:31.123750 (+ 2146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:31.123756 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:31.123859 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:31.123943 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:31.125387 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:31.126416 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:31.128642 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:31.129283 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:31.129418 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:31.130989 (+ 1571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:31.130995 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:31.131001 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:52:31.131022 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:31.131063 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:52:31.131077 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:31.131162 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:31.131457 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 03:52:31.131571 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 03:52:31.133066 8713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:31.120294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37853 (local address 127.26.146.130:41631)
0808 03:52:31.121398 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:31.121458 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:31.123480 (+ 2022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:31.124248 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:31.124259 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:31.124374 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:31.125174 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:31.125208 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:31.126679 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:31.126684 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:31.128478 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:31.128503 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:31.129547 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:31.129554 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:31.129825 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:31.130629 (+ 804us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:52:31.130632 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:31.130682 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:52:31.130784 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:52:31.131178 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:52:31.131188 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:52:31.131196 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 03:52:31.131392 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 03:52:31.131785 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":299,"threads_started":1}
I20260808 03:52:31.134369 8711 heartbeater.cc:347] Connected to a master server at 127.26.146.190:37853
I20260808 03:52:31.134697 8711 heartbeater.cc:464] Registering TS with master...
I20260808 03:52:31.135568 8711 heartbeater.cc:511] Master 127.26.146.190:37853 requested a full tablet report, sending...
I20260808 03:52:31.137053 8267 ts_manager.cc:194] Registered new tserver with Master: 6c36bb55593742ef8aea3b34feccf9d7 (127.26.146.130:42813)
I20260808 03:52:31.137844 8267 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.146.130:41631
W20260808 03:52:31.455606 8714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:31.455952 8714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:31.456099 8714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:31.463598 8714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:31.463696 8714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:31.463879 8714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:52:31.472754 8714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:41515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:37853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:31.473976 8714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:31.475711 8714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:31.488131 8724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:31.488201 8725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:31.488298 8729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:31.489437 8714 server_base.cc:1034] running on GCE node
I20260808 03:52:31.490051 8714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:31.490906 8714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:31.492122 8714 hybrid_clock.cc:648] HybridClock initialized: now 1786161151492061 us; error 44 us; skew 500 ppm
I20260808 03:52:31.492440 8714 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:52:31.494582 8714 webserver.cc:460] Webserver started at http://127.26.146.131:35783/ using document root <none> and password file <none>
I20260808 03:52:31.495244 8714 fs_manager.cc:359] Metadata directory not provided
I20260808 03:52:31.495347 8714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:52:31.495667 8714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:52:31.498549 8714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "c538cc64d5254b17ab4ac37aa28331e3"
format_stamp: "Formatted at 2026-08-08 03:52:31 on dist-test-slave-jlzn"
server_key: "e08d4402d7a8fbeb336b810cdc43afbb"
server_key_iv: "ac11b652db7a87fd4e37a1be3887ed37"
server_key_version: "encryptionkey@0"
I20260808 03:52:31.499384 8714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "c538cc64d5254b17ab4ac37aa28331e3"
format_stamp: "Formatted at 2026-08-08 03:52:31 on dist-test-slave-jlzn"
server_key: "e08d4402d7a8fbeb336b810cdc43afbb"
server_key_iv: "ac11b652db7a87fd4e37a1be3887ed37"
server_key_version: "encryptionkey@0"
I20260808 03:52:31.504451 8714 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260808 03:52:31.507732 8740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:31.508965 8714 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 03:52:31.509125 8714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "c538cc64d5254b17ab4ac37aa28331e3"
format_stamp: "Formatted at 2026-08-08 03:52:31 on dist-test-slave-jlzn"
server_key: "e08d4402d7a8fbeb336b810cdc43afbb"
server_key_iv: "ac11b652db7a87fd4e37a1be3887ed37"
server_key_version: "encryptionkey@0"
I20260808 03:52:31.509315 8714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:52:31.529418 8714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:52:31.530241 8714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:52:31.530529 8714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:52:31.531471 8714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:52:31.532948 8714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:52:31.533049 8714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:31.533160 8714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:52:31.533234 8714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:31.562901 8714 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:32897
I20260808 03:52:31.562953 8904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:32897 every 8 connection(s)
I20260808 03:52:31.564157 8714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:52:31.572299 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 8714
I20260808 03:52:31.572448 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:52:31.572885 27210 external_mini_cluster.cc:1428] Setting key caa76e28fd82d1c11941ab26f6698591
I20260808 03:52:31.576257 8535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:31.566518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:42141 (local address 127.26.146.190:37853)
0808 03:52:31.566687 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:31.566692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:31.568154 (+ 1462us) server_negotiation.cc:407] Connection header received
0808 03:52:31.569889 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:31.569896 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:31.569972 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:31.570057 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:31.571081 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:31.571862 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:31.573465 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:31.574021 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:31.574131 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:31.575484 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:31.575490 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:31.575493 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:52:31.575511 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:31.575549 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:52:31.575565 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:31.575654 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:31.575927 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 03:52:31.575999 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 03:52:31.577340 8907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:31.567017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:37853 (local address 127.26.146.131:42141)
0808 03:52:31.568005 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:31.568040 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:31.569628 (+ 1588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:31.570297 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:31.570306 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:31.570384 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:31.570920 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:31.570936 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:31.572024 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:31.572028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:31.573329 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:31.573342 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:31.574245 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:31.574250 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:31.574448 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:31.575180 (+ 732us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:52:31.575183 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:31.575230 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:52:31.575307 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:52:31.575689 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:52:31.575698 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:52:31.575704 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 03:52:31.575913 (+ 209us) client_negotiation.cc:240] Negotiation successful
0808 03:52:31.576264 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":261,"threads_started":1}
I20260808 03:52:31.578512 8905 heartbeater.cc:347] Connected to a master server at 127.26.146.190:37853
I20260808 03:52:31.578778 8905 heartbeater.cc:464] Registering TS with master...
I20260808 03:52:31.579486 8905 heartbeater.cc:511] Master 127.26.146.190:37853 requested a full tablet report, sending...
I20260808 03:52:31.580792 8267 ts_manager.cc:194] Registered new tserver with Master: c538cc64d5254b17ab4ac37aa28331e3 (127.26.146.131:32897)
I20260808 03:52:31.581557 8267 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.146.131:42141
I20260808 03:52:31.587947 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 03:52:31.593767 8535 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:31.592169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43230 (local address 127.26.146.190:37853)
0808 03:52:31.592315 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:31.592321 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:31.592756 (+ 435us) server_negotiation.cc:407] Connection header received
0808 03:52:31.592909 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:31.592914 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:31.592986 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:31.593065 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:31.593512 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43230: BlockingRecv error: recv got EOF from 127.0.0.1:43230 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 03:52:31.594290 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 8357
I20260808 03:52:31.602051 8923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:31.595140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39934 (local address 127.26.146.129:38121)
0808 03:52:31.595643 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:31.595649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:31.595676 (+ 27us) server_negotiation.cc:407] Connection header received
0808 03:52:31.595732 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:31.595737 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:31.595997 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:31.596113 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:31.596565 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:31.597855 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:31.599122 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:31.599720 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:31.599851 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:31.600827 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:31.600836 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:31.600848 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:52:31.601404 (+ 556us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:31.601449 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:52:31.601468 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:31.601554 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:31.601654 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 03:52:31.601769 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":243,"threads_started":1}
I20260808 03:52:31.603197 8525 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:31.683866 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 8357
I20260808 03:52:31.705919 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 8537
I20260808 03:52:31.713474 8925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:31.706820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41702 (local address 127.26.146.130:42813)
0808 03:52:31.707390 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:31.707397 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:31.707425 (+ 28us) server_negotiation.cc:407] Connection header received
0808 03:52:31.707487 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:31.707494 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:31.707738 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:31.707914 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:31.708230 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:31.709248 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:31.710422 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:31.710987 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:31.711092 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:31.712080 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:31.712088 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:31.712102 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:52:31.712756 (+ 654us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:31.712815 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:52:31.712837 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:31.712941 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:31.713048 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 03:52:31.713158 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":295,"threads_started":1}
I20260808 03:52:31.714759 8706 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:31.794695 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 8537
I20260808 03:52:31.818945 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 8714
I20260808 03:52:31.826912 8929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:31.820002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43966 (local address 127.26.146.131:32897)
0808 03:52:31.820666 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:31.820674 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:31.820706 (+ 32us) server_negotiation.cc:407] Connection header received
0808 03:52:31.820775 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:31.820781 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:31.820995 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:31.821118 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:31.821572 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:31.822622 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:31.823788 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:31.824388 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:31.824486 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:31.825500 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:31.825508 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:31.825521 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:52:31.826119 (+ 598us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:31.826167 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:52:31.826186 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:31.826271 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:31.826420 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 03:52:31.826537 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":332,"threads_started":1}
I20260808 03:52:31.828195 8896 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:31.913142 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 8714
I20260808 03:52:31.940210 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 8225
I20260808 03:52:31.941457 8310 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:32.045058 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 8225
2026-08-08T03:52:32Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2417 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T03:52:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:52:32Z Disabled control of system clock
I20260808 03:52:32.126801 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:38583
--webserver_interface=127.26.146.190
--webserver_port=0
--builtin_ntp_servers=127.26.146.148:35519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:38583
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 03:52:32.457479 8937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:32.457772 8937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:32.457862 8937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:32.464845 8937 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:52:32.464942 8937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:32.464986 8937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:32.465042 8937 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:52:32.465091 8937 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:52:32.473940 8937 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:35519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:38583
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:38583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:32.475131 8937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:32.476898 8937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:32.488333 8943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:32.488253 8945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:32.488291 8950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:32.489306 8937 server_base.cc:1034] running on GCE node
I20260808 03:52:32.490041 8937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:32.491371 8937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:32.492588 8937 hybrid_clock.cc:648] HybridClock initialized: now 1786161152492550 us; error 32 us; skew 500 ppm
I20260808 03:52:32.492929 8937 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:52:32.495091 8937 webserver.cc:460] Webserver started at http://127.26.146.190:37821/ using document root <none> and password file <none>
I20260808 03:52:32.495749 8937 fs_manager.cc:359] Metadata directory not provided
I20260808 03:52:32.495877 8937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:52:32.496184 8937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:52:32.498989 8937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "90c5b1616fce4c1296b74becefe294fd"
format_stamp: "Formatted at 2026-08-08 03:52:32 on dist-test-slave-jlzn"
server_key: "c8b85cca220f9baa66b097ef6488fad4"
server_key_iv: "acb8ab18ac0c05903e2b78355cd13f40"
server_key_version: "encryptionkey@0"
I20260808 03:52:32.499752 8937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "90c5b1616fce4c1296b74becefe294fd"
format_stamp: "Formatted at 2026-08-08 03:52:32 on dist-test-slave-jlzn"
server_key: "c8b85cca220f9baa66b097ef6488fad4"
server_key_iv: "acb8ab18ac0c05903e2b78355cd13f40"
server_key_version: "encryptionkey@0"
I20260808 03:52:32.504770 8937 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260808 03:52:32.508050 8960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:32.509375 8937 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 03:52:32.509552 8937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "90c5b1616fce4c1296b74becefe294fd"
format_stamp: "Formatted at 2026-08-08 03:52:32 on dist-test-slave-jlzn"
server_key: "c8b85cca220f9baa66b097ef6488fad4"
server_key_iv: "acb8ab18ac0c05903e2b78355cd13f40"
server_key_version: "encryptionkey@0"
I20260808 03:52:32.509763 8937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:52:32.515372 8937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:52:32.516146 8937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:52:32.516418 8937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:52:32.533026 8937 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:38583
I20260808 03:52:32.533048 9031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:38583 every 8 connection(s)
I20260808 03:52:32.534296 8937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:52:32.537853 9033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:52:32.539012 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 8937
I20260808 03:52:32.539249 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:52:32.540076 27210 external_mini_cluster.cc:1428] Setting key e29276e00825b1804c9abdc54ea2d0fe
I20260808 03:52:32.548207 9038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:32.542116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35820 (local address 127.26.146.190:38583)
0808 03:52:32.543166 (+ 1050us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:32.543184 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:32.543227 (+ 43us) server_negotiation.cc:407] Connection header received
0808 03:52:32.544852 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:32.544880 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:32.545277 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:32.545592 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:32.546630 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:32.546636 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:32.546649 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:52:32.546662 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:32.546725 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:52:32.546740 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:32.546904 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:32.547001 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 03:52:32.547248 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":390,"threads_started":1}
I20260808 03:52:32.551234 9033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90c5b1616fce4c1296b74becefe294fd: Bootstrap starting.
I20260808 03:52:32.554044 9033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90c5b1616fce4c1296b74becefe294fd: Neither blocks nor log segments found. Creating new log.
I20260808 03:52:32.554981 9033 log.cc:824] T 00000000000000000000000000000000 P 90c5b1616fce4c1296b74becefe294fd: Log is configured to *not* fsync() on all Append() calls
I20260808 03:52:32.557655 9033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90c5b1616fce4c1296b74becefe294fd: No bootstrap required, opened a new log
I20260808 03:52:32.564538 9033 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90c5b1616fce4c1296b74becefe294fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c5b1616fce4c1296b74becefe294fd" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38583 } }
I20260808 03:52:32.564855 9033 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90c5b1616fce4c1296b74becefe294fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:52:32.564949 9033 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90c5b1616fce4c1296b74becefe294fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90c5b1616fce4c1296b74becefe294fd, State: Initialized, Role: FOLLOWER
I20260808 03:52:32.565460 9033 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90c5b1616fce4c1296b74becefe294fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c5b1616fce4c1296b74becefe294fd" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38583 } }
I20260808 03:52:32.565663 9033 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90c5b1616fce4c1296b74becefe294fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:52:32.565799 9033 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90c5b1616fce4c1296b74becefe294fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:52:32.565950 9033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90c5b1616fce4c1296b74becefe294fd [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:52:32.567966 9033 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90c5b1616fce4c1296b74becefe294fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c5b1616fce4c1296b74becefe294fd" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38583 } }
I20260808 03:52:32.568480 9033 leader_election.cc:302] T 00000000000000000000000000000000 P 90c5b1616fce4c1296b74becefe294fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90c5b1616fce4c1296b74becefe294fd; no voters:
I20260808 03:52:32.569005 9033 leader_election.cc:288] T 00000000000000000000000000000000 P 90c5b1616fce4c1296b74becefe294fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:52:32.569137 9040 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90c5b1616fce4c1296b74becefe294fd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:52:32.569778 9040 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90c5b1616fce4c1296b74becefe294fd [term 1 LEADER]: Becoming Leader. State: Replica: 90c5b1616fce4c1296b74becefe294fd, State: Running, Role: LEADER
I20260808 03:52:32.570345 9040 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90c5b1616fce4c1296b74becefe294fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c5b1616fce4c1296b74becefe294fd" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38583 } }
I20260808 03:52:32.570905 9033 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90c5b1616fce4c1296b74becefe294fd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:52:32.572947 9042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90c5b1616fce4c1296b74becefe294fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90c5b1616fce4c1296b74becefe294fd. Latest consensus state: current_term: 1 leader_uuid: "90c5b1616fce4c1296b74becefe294fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c5b1616fce4c1296b74becefe294fd" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38583 } } }
I20260808 03:52:32.572894 9041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90c5b1616fce4c1296b74becefe294fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90c5b1616fce4c1296b74becefe294fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c5b1616fce4c1296b74becefe294fd" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38583 } } }
I20260808 03:52:32.573341 9042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90c5b1616fce4c1296b74becefe294fd [sys.catalog]: This master's current role is: LEADER
I20260808 03:52:32.573350 9041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90c5b1616fce4c1296b74becefe294fd [sys.catalog]: This master's current role is: LEADER
I20260808 03:52:32.576757 9052 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:52:32.581601 9052 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:52:32.590372 9052 catalog_manager.cc:1347] Generated new cluster ID: e514779a70b34e3e9abfc9106dc90313
I20260808 03:52:32.590471 9052 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:52:32.614360 9052 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:52:32.614787 9052 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:52:32.623677 9052 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90c5b1616fce4c1296b74becefe294fd: Generated new TSK 0
I20260808 03:52:32.624460 9052 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:52:32.635557 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:38583
--builtin_ntp_servers=127.26.146.148:35519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 03:52:32.969976 9071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:32.970245 9071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:32.970371 9071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:32.977608 9071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:32.977702 9071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:32.977857 9071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:52:32.986584 9071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:35519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:38583
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:32.987884 9071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:32.989634 9071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:33.002063 9082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:33.002139 9078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:33.002066 9080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:33.002898 9071 server_base.cc:1034] running on GCE node
I20260808 03:52:33.003489 9071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:33.004457 9071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:33.005661 9071 hybrid_clock.cc:648] HybridClock initialized: now 1786161153005607 us; error 50 us; skew 500 ppm
I20260808 03:52:33.005959 9071 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:52:33.008147 9071 webserver.cc:460] Webserver started at http://127.26.146.129:38343/ using document root <none> and password file <none>
I20260808 03:52:33.008792 9071 fs_manager.cc:359] Metadata directory not provided
I20260808 03:52:33.008893 9071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:52:33.009193 9071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:52:33.012135 9071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "a6ebd243f8494ad4b560cd87c872c857"
format_stamp: "Formatted at 2026-08-08 03:52:33 on dist-test-slave-jlzn"
server_key: "0b7fcae5f3c76464eba0cace0322ad47"
server_key_iv: "660f21ccb7fda35237bfa05a2d69b749"
server_key_version: "encryptionkey@0"
I20260808 03:52:33.012966 9071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "a6ebd243f8494ad4b560cd87c872c857"
format_stamp: "Formatted at 2026-08-08 03:52:33 on dist-test-slave-jlzn"
server_key: "0b7fcae5f3c76464eba0cace0322ad47"
server_key_iv: "660f21ccb7fda35237bfa05a2d69b749"
server_key_version: "encryptionkey@0"
I20260808 03:52:33.018106 9071 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 03:52:33.021423 9089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:33.022686 9071 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 03:52:33.022835 9071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "a6ebd243f8494ad4b560cd87c872c857"
format_stamp: "Formatted at 2026-08-08 03:52:33 on dist-test-slave-jlzn"
server_key: "0b7fcae5f3c76464eba0cace0322ad47"
server_key_iv: "660f21ccb7fda35237bfa05a2d69b749"
server_key_version: "encryptionkey@0"
I20260808 03:52:33.023028 9071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:52:33.028549 9071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:52:33.029282 9071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:52:33.029532 9071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:52:33.030381 9071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:52:33.031742 9071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:52:33.031900 9071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260808 03:52:33.032006 9071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:52:33.032078 9071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:33.061987 9071 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:40875
I20260808 03:52:33.062033 9256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:40875 every 8 connection(s)
I20260808 03:52:33.063257 9071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:52:33.070776 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:33.065746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:45687 (local address 127.26.146.190:38583)
0808 03:52:33.066158 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:33.066163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:33.067276 (+ 1113us) server_negotiation.cc:407] Connection header received
0808 03:52:33.068906 (+ 1630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:33.068911 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:33.068963 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:33.069058 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:33.070191 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:33.070194 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:33.070197 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:52:33.070212 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:33.070253 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:52:33.070265 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:33.070354 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:33.070493 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 03:52:33.070554 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":213,"threads_started":1}
I20260808 03:52:33.071624 9264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:33.066280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38583 (local address 127.26.146.129:45687)
0808 03:52:33.067116 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:33.067150 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:33.068669 (+ 1519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:33.069259 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:33.069267 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:33.069350 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:33.069924 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:52:33.069927 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:33.069961 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:52:33.070040 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:52:33.070336 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:52:33.070341 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:52:33.070346 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 03:52:33.070552 (+ 206us) client_negotiation.cc:240] Negotiation successful
0808 03:52:33.070736 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":243,"threads_started":1}
I20260808 03:52:33.072849 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 9071
I20260808 03:52:33.072997 9260 heartbeater.cc:347] Connected to a master server at 127.26.146.190:38583
I20260808 03:52:33.073041 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:52:33.073274 9260 heartbeater.cc:464] Registering TS with master...
I20260808 03:52:33.073658 27210 external_mini_cluster.cc:1428] Setting key 2155e0cfd9ed4e4ec18ae0e42908876d
I20260808 03:52:33.073884 9260 heartbeater.cc:511] Master 127.26.146.190:38583 requested a full tablet report, sending...
I20260808 03:52:33.075860 8979 ts_manager.cc:194] Registered new tserver with Master: a6ebd243f8494ad4b560cd87c872c857 (127.26.146.129:40875)
I20260808 03:52:33.076363 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:38583
--builtin_ntp_servers=127.26.146.148:35519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 03:52:33.401681 9266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:33.401952 9266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:33.402077 9266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:33.409103 9266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:33.409209 9266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:33.409356 9266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:52:33.418238 9266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:35519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:38583
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:33.419445 9266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:33.421298 9266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:33.433841 9275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:33.433917 9276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:33.433892 9278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:33.434937 9266 server_base.cc:1034] running on GCE node
I20260808 03:52:33.435506 9266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:33.436422 9266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:33.437624 9266 hybrid_clock.cc:648] HybridClock initialized: now 1786161153437572 us; error 37 us; skew 500 ppm
I20260808 03:52:33.437917 9266 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:52:33.440120 9266 webserver.cc:460] Webserver started at http://127.26.146.130:32839/ using document root <none> and password file <none>
I20260808 03:52:33.440776 9266 fs_manager.cc:359] Metadata directory not provided
I20260808 03:52:33.440877 9266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:52:33.441176 9266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:52:33.443943 9266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "0ed94cc9e5a24d7d8693e042f7797a58"
format_stamp: "Formatted at 2026-08-08 03:52:33 on dist-test-slave-jlzn"
server_key: "7606e4ecfdcd2431e51c860a1371f080"
server_key_iv: "7bd2604971e3df55a41a501d01ee9784"
server_key_version: "encryptionkey@0"
I20260808 03:52:33.444697 9266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "0ed94cc9e5a24d7d8693e042f7797a58"
format_stamp: "Formatted at 2026-08-08 03:52:33 on dist-test-slave-jlzn"
server_key: "7606e4ecfdcd2431e51c860a1371f080"
server_key_iv: "7bd2604971e3df55a41a501d01ee9784"
server_key_version: "encryptionkey@0"
I20260808 03:52:33.449489 9266 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 03:52:33.452641 9286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:33.453826 9266 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 03:52:33.453974 9266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "0ed94cc9e5a24d7d8693e042f7797a58"
format_stamp: "Formatted at 2026-08-08 03:52:33 on dist-test-slave-jlzn"
server_key: "7606e4ecfdcd2431e51c860a1371f080"
server_key_iv: "7bd2604971e3df55a41a501d01ee9784"
server_key_version: "encryptionkey@0"
I20260808 03:52:33.454166 9266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:52:33.459494 9266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:52:33.460283 9266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:52:33.460525 9266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:52:33.461488 9266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:52:33.462829 9266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:52:33.462929 9266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:33.463033 9266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:52:33.463111 9266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:33.491652 9266 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:34447
I20260808 03:52:33.491704 9454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:34447 every 8 connection(s)
I20260808 03:52:33.492928 9266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:52:33.500362 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:33.495329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:49149 (local address 127.26.146.190:38583)
0808 03:52:33.495511 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:33.495516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:33.496873 (+ 1357us) server_negotiation.cc:407] Connection header received
0808 03:52:33.498472 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:33.498476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:33.498530 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:33.498623 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:33.499748 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:33.499752 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:33.499755 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:52:33.499770 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:33.499841 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:52:33.499857 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:33.499947 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:33.500078 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 03:52:33.500135 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 03:52:33.501207 9459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:33.495862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38583 (local address 127.26.146.130:49149)
0808 03:52:33.496740 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:33.496774 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:33.498240 (+ 1466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:33.498807 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:33.498815 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:33.498898 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:33.499481 (+ 583us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:52:33.499484 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:33.499518 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:52:33.499597 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:52:33.499924 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:52:33.499932 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:52:33.499937 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 03:52:33.500142 (+ 205us) client_negotiation.cc:240] Negotiation successful
0808 03:52:33.500321 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":272,"threads_started":1}
I20260808 03:52:33.501235 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 9266
I20260808 03:52:33.501396 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:52:33.501833 27210 external_mini_cluster.cc:1428] Setting key 5c2ccec6d7e70e1bcf36ac20395bdaaa
I20260808 03:52:33.502527 9456 heartbeater.cc:347] Connected to a master server at 127.26.146.190:38583
I20260808 03:52:33.502797 9456 heartbeater.cc:464] Registering TS with master...
I20260808 03:52:33.503351 9456 heartbeater.cc:511] Master 127.26.146.190:38583 requested a full tablet report, sending...
I20260808 03:52:33.504652 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:38583
--builtin_ntp_servers=127.26.146.148:35519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 03:52:33.504693 8979 ts_manager.cc:194] Registered new tserver with Master: 0ed94cc9e5a24d7d8693e042f7797a58 (127.26.146.130:34447)
W20260808 03:52:33.835618 9461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:33.835942 9461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:33.836089 9461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:33.843129 9461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:33.843236 9461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:33.843389 9461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:52:33.852272 9461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:35519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:38583
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:33.853530 9461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:33.855248 9461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:33.868116 9470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:33.868327 9473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:33.868898 9468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:33.870254 9461 server_base.cc:1034] running on GCE node
I20260808 03:52:33.870944 9461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:33.871966 9461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:33.873167 9461 hybrid_clock.cc:648] HybridClock initialized: now 1786161153873111 us; error 49 us; skew 500 ppm
I20260808 03:52:33.873466 9461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:52:33.875648 9461 webserver.cc:460] Webserver started at http://127.26.146.131:34689/ using document root <none> and password file <none>
I20260808 03:52:33.876349 9461 fs_manager.cc:359] Metadata directory not provided
I20260808 03:52:33.876454 9461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:52:33.876772 9461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:52:33.879551 9461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "8681a8132ea9400ab6911391b62ba4d0"
format_stamp: "Formatted at 2026-08-08 03:52:33 on dist-test-slave-jlzn"
server_key: "7b3b0edefe58a05e6027086458b1b85e"
server_key_iv: "4b7ec581e730783e9c572cc6c1482181"
server_key_version: "encryptionkey@0"
I20260808 03:52:33.880391 9461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "8681a8132ea9400ab6911391b62ba4d0"
format_stamp: "Formatted at 2026-08-08 03:52:33 on dist-test-slave-jlzn"
server_key: "7b3b0edefe58a05e6027086458b1b85e"
server_key_iv: "4b7ec581e730783e9c572cc6c1482181"
server_key_version: "encryptionkey@0"
I20260808 03:52:33.885330 9461 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 03:52:33.888597 9482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:33.889892 9461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:52:33.890044 9461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "8681a8132ea9400ab6911391b62ba4d0"
format_stamp: "Formatted at 2026-08-08 03:52:33 on dist-test-slave-jlzn"
server_key: "7b3b0edefe58a05e6027086458b1b85e"
server_key_iv: "4b7ec581e730783e9c572cc6c1482181"
server_key_version: "encryptionkey@0"
I20260808 03:52:33.890261 9461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:52:33.895783 9461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:52:33.896538 9461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:52:33.896818 9461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:52:33.897673 9461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:52:33.899020 9461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:52:33.899116 9461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:33.899224 9461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:52:33.899295 9461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:33.929184 9461 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:33269
I20260808 03:52:33.929236 9635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:33269 every 8 connection(s)
I20260808 03:52:33.930478 9461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:52:33.938056 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:33.932953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:51453 (local address 127.26.146.190:38583)
0808 03:52:33.933134 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:33.933139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:33.934503 (+ 1364us) server_negotiation.cc:407] Connection header received
0808 03:52:33.936175 (+ 1672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:33.936180 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:33.936235 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:33.936326 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:33.937520 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:33.937523 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:33.937526 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:52:33.937538 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:33.937563 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:52:33.937573 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:33.937665 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:33.937799 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 03:52:33.937849 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 03:52:33.938912 9639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:33.933473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38583 (local address 127.26.146.131:51453)
0808 03:52:33.934344 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:33.934383 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:33.935904 (+ 1521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:33.936505 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:33.936513 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:33.936599 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:33.937267 (+ 668us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:52:33.937269 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:33.937308 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:52:33.937393 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:52:33.937648 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:52:33.937654 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:52:33.937659 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 03:52:33.937860 (+ 201us) client_negotiation.cc:240] Negotiation successful
0808 03:52:33.938044 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":261,"threads_started":1}
I20260808 03:52:33.939862 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 9461
I20260808 03:52:33.940050 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:52:33.940269 9637 heartbeater.cc:347] Connected to a master server at 127.26.146.190:38583
I20260808 03:52:33.940596 9637 heartbeater.cc:464] Registering TS with master...
I20260808 03:52:33.940591 27210 external_mini_cluster.cc:1428] Setting key 511124f4d4728a744a0d224e729b9274
I20260808 03:52:33.941265 9637 heartbeater.cc:511] Master 127.26.146.190:38583 requested a full tablet report, sending...
I20260808 03:52:33.942653 8979 ts_manager.cc:194] Registered new tserver with Master: 8681a8132ea9400ab6911391b62ba4d0 (127.26.146.131:33269)
I20260808 03:52:33.943562 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 03:52:33.949733 9262 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:52:33.948140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35836 (local address 127.26.146.190:38583)
0808 03:52:33.948290 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:33.948295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:33.948843 (+ 548us) server_negotiation.cc:407] Connection header received
0808 03:52:33.948993 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:33.948999 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:33.949053 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:33.949140 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:33.949491 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35836: BlockingRecv error: recv got EOF from 127.0.0.1:35836 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 03:52:33.950302 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 9071
I20260808 03:52:33.953536 9651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:33.951194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34138 (local address 127.26.146.129:40875)
0808 03:52:33.951701 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:33.951715 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:33.951739 (+ 24us) server_negotiation.cc:407] Connection header received
0808 03:52:33.951794 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:33.951827 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:33.952016 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:33.952143 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:33.952944 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:33.952950 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:33.952959 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:52:33.952977 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:33.953012 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:52:33.953026 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:33.953132 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:33.953180 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 03:52:33.953255 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":242,"threads_started":1}
I20260808 03:52:33.954624 9248 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:34.033267 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 9071
I20260808 03:52:34.054226 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 9266
I20260808 03:52:34.057880 9655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:34.055189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39708 (local address 127.26.146.130:34447)
0808 03:52:34.055647 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:34.055661 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:34.055688 (+ 27us) server_negotiation.cc:407] Connection header received
0808 03:52:34.055761 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:34.055767 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:34.056016 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:34.056162 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:34.057126 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:34.057133 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:34.057145 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:52:34.057164 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:34.057206 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:52:34.057220 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:34.057339 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:34.057510 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 03:52:34.057613 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":200,"threads_started":1}
I20260808 03:52:34.059103 9449 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:34.138761 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 9266
I20260808 03:52:34.160318 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 9461
I20260808 03:52:34.163931 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:34.161281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41290 (local address 127.26.146.131:33269)
0808 03:52:34.161910 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:34.161917 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:34.161949 (+ 32us) server_negotiation.cc:407] Connection header received
0808 03:52:34.162013 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:34.162020 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:34.162235 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:34.162358 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:34.163211 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:34.163218 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:34.163229 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:52:34.163241 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:34.163281 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:52:34.163294 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:34.163424 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:34.163470 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 03:52:34.163568 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":298,"threads_started":1}
I20260808 03:52:34.165112 9630 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:34.244104 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 9461
I20260808 03:52:34.269224 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 8937
I20260808 03:52:34.270586 9027 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:34.371665 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 8937
2026-08-08T03:52:34Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2329 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:52:34 dist-test-slave-jlzn krb5kdc[9665](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:52:34 dist-test-slave-jlzn krb5kdc[9665](info): set up 2 sockets
Aug 08 03:52:34 dist-test-slave-jlzn krb5kdc[9665](info): commencing operation
krb5kdc: starting...
W20260808 03:52:35.558784 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:52:35 dist-test-slave-jlzn krb5kdc[9665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:52:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:52:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:52:35.823035 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:42635
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:46875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:42635
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:52:36.151243 9682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:36.151548 9682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:36.151681 9682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:36.158762 9682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:52:36.158861 9682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:36.158936 9682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:36.159019 9682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:52:36.159077 9682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:52:36.168776 9682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:46875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:42635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:42635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:36.170034 9682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:36.171329 9682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:36.182578 9692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:36.182679 9689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:36.182648 9690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:36.183457 9682 server_base.cc:1034] running on GCE node
I20260808 03:52:36.184279 9682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:36.185761 9682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:36.186973 9682 hybrid_clock.cc:648] HybridClock initialized: now 1786161156186930 us; error 31 us; skew 500 ppm
I20260808 03:52:36.187318 9682 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:52:36 dist-test-slave-jlzn krb5kdc[9665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161156, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:52:36.194566 9682 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:52:36.195977 9682 webserver.cc:460] Webserver started at http://127.26.146.190:44469/ using document root <none> and password file <none>
I20260808 03:52:36.196671 9682 fs_manager.cc:359] Metadata directory not provided
I20260808 03:52:36.196775 9682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:52:36.197043 9682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:52:36.200104 9682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "d46ace43fb91402881819090fef1355d"
format_stamp: "Formatted at 2026-08-08 03:52:36 on dist-test-slave-jlzn"
server_key: "ee370f5efb12b35368610f6cab93f3d6"
server_key_iv: "ba6f24c835d4b9f8c1dd18c9117244ab"
server_key_version: "encryptionkey@0"
I20260808 03:52:36.200841 9682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "d46ace43fb91402881819090fef1355d"
format_stamp: "Formatted at 2026-08-08 03:52:36 on dist-test-slave-jlzn"
server_key: "ee370f5efb12b35368610f6cab93f3d6"
server_key_iv: "ba6f24c835d4b9f8c1dd18c9117244ab"
server_key_version: "encryptionkey@0"
I20260808 03:52:36.205821 9682 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:52:36.209242 9701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:36.210512 9682 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:52:36.210675 9682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "d46ace43fb91402881819090fef1355d"
format_stamp: "Formatted at 2026-08-08 03:52:36 on dist-test-slave-jlzn"
server_key: "ee370f5efb12b35368610f6cab93f3d6"
server_key_iv: "ba6f24c835d4b9f8c1dd18c9117244ab"
server_key_version: "encryptionkey@0"
I20260808 03:52:36.210852 9682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:52:36.235304 9682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:52:36.239336 9682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:52:36.239603 9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:52:36.256906 9682 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:42635
I20260808 03:52:36.256938 9787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:42635 every 8 connection(s)
I20260808 03:52:36.258193 9682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:52:36.261915 9790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:52:36.267776 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 9682
I20260808 03:52:36.268026 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:52:36.268661 27210 external_mini_cluster.cc:1428] Setting key c41d2574d1389979424b254681b9d9fc
I20260808 03:52:36.272982 9790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d46ace43fb91402881819090fef1355d: Bootstrap starting.
I20260808 03:52:36.276897 9790 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d46ace43fb91402881819090fef1355d: Neither blocks nor log segments found. Creating new log.
I20260808 03:52:36.278195 9790 log.cc:824] T 00000000000000000000000000000000 P d46ace43fb91402881819090fef1355d: Log is configured to *not* fsync() on all Append() calls
I20260808 03:52:36.281517 9790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d46ace43fb91402881819090fef1355d: No bootstrap required, opened a new log
Aug 08 03:52:36 dist-test-slave-jlzn krb5kdc[9665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:52:36.290994 9790 raft_consensus.cc:348] T 00000000000000000000000000000000 P d46ace43fb91402881819090fef1355d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46ace43fb91402881819090fef1355d" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 42635 } }
I20260808 03:52:36.291522 9790 raft_consensus.cc:374] T 00000000000000000000000000000000 P d46ace43fb91402881819090fef1355d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:52:36.291649 9790 raft_consensus.cc:729] T 00000000000000000000000000000000 P d46ace43fb91402881819090fef1355d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d46ace43fb91402881819090fef1355d, State: Initialized, Role: FOLLOWER
I20260808 03:52:36.292274 9790 consensus_queue.cc:260] T 00000000000000000000000000000000 P d46ace43fb91402881819090fef1355d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46ace43fb91402881819090fef1355d" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 42635 } }
I20260808 03:52:36.292495 9790 raft_consensus.cc:388] T 00000000000000000000000000000000 P d46ace43fb91402881819090fef1355d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:52:36.292629 9790 raft_consensus.cc:482] T 00000000000000000000000000000000 P d46ace43fb91402881819090fef1355d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:52:36.292815 9790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d46ace43fb91402881819090fef1355d [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:52:36.292812 9793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:36.270638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41818 (local address 127.26.146.190:42635)
0808 03:52:36.271430 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:36.271446 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:36.271516 (+ 70us) server_negotiation.cc:407] Connection header received
0808 03:52:36.273818 (+ 2302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:36.273847 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:36.274307 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:36.274743 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:36.276060 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:36.278018 (+ 1958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:36.279587 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:36.280607 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:36.280795 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:36.284493 (+ 3698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:36.284504 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:36.284526 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:36.284570 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:36.288501 (+ 3931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:36.289157 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:36.289170 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:36.289178 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:36.289407 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:36.289716 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:36.289721 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:36.289725 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:36.290436 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:36.290752 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:36.291018 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 03:52:36.291460 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":320,"threads_started":1}
I20260808 03:52:36.295177 9790 raft_consensus.cc:504] T 00000000000000000000000000000000 P d46ace43fb91402881819090fef1355d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46ace43fb91402881819090fef1355d" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 42635 } }
I20260808 03:52:36.295720 9790 leader_election.cc:302] T 00000000000000000000000000000000 P d46ace43fb91402881819090fef1355d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d46ace43fb91402881819090fef1355d; no voters:
I20260808 03:52:36.296283 9790 leader_election.cc:288] T 00000000000000000000000000000000 P d46ace43fb91402881819090fef1355d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:52:36.296404 9799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d46ace43fb91402881819090fef1355d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:52:36.297528 9799 raft_consensus.cc:686] T 00000000000000000000000000000000 P d46ace43fb91402881819090fef1355d [term 1 LEADER]: Becoming Leader. State: Replica: d46ace43fb91402881819090fef1355d, State: Running, Role: LEADER
I20260808 03:52:36.298112 9799 consensus_queue.cc:237] T 00000000000000000000000000000000 P d46ace43fb91402881819090fef1355d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46ace43fb91402881819090fef1355d" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 42635 } }
I20260808 03:52:36.298345 9790 sys_catalog.cc:564] T 00000000000000000000000000000000 P d46ace43fb91402881819090fef1355d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:52:36.300915 9801 sys_catalog.cc:455] T 00000000000000000000000000000000 P d46ace43fb91402881819090fef1355d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d46ace43fb91402881819090fef1355d. Latest consensus state: current_term: 1 leader_uuid: "d46ace43fb91402881819090fef1355d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46ace43fb91402881819090fef1355d" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 42635 } } }
I20260808 03:52:36.301256 9801 sys_catalog.cc:458] T 00000000000000000000000000000000 P d46ace43fb91402881819090fef1355d [sys.catalog]: This master's current role is: LEADER
I20260808 03:52:36.301730 9800 sys_catalog.cc:455] T 00000000000000000000000000000000 P d46ace43fb91402881819090fef1355d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d46ace43fb91402881819090fef1355d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46ace43fb91402881819090fef1355d" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 42635 } } }
I20260808 03:52:36.302062 9800 sys_catalog.cc:458] T 00000000000000000000000000000000 P d46ace43fb91402881819090fef1355d [sys.catalog]: This master's current role is: LEADER
I20260808 03:52:36.303434 9808 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:52:36.307943 9808 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:52:36.316699 9808 catalog_manager.cc:1347] Generated new cluster ID: e983f22f41d74b63a9eaf276b6bbeac3
I20260808 03:52:36.316812 9808 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:52:36.329634 9808 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:52:36.330936 9808 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:52:36.339628 9808 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d46ace43fb91402881819090fef1355d: Generated new TSK 0
I20260808 03:52:36.340432 9808 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
I20260808 03:52:36.409643 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:42635
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:46875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:52:36.738430 9826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:36.738736 9826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:36.738878 9826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:36.746235 9826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:36.746363 9826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:36.746522 9826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:52:36.756119 9826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:46875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:42635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:36.757442 9826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:36.758757 9826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:36.771924 9832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:36.771878 9834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:36.771944 9831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:36.777298 9826 server_base.cc:1034] running on GCE node
I20260808 03:52:36.777935 9826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:36.778904 9826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:36.780112 9826 hybrid_clock.cc:648] HybridClock initialized: now 1786161156780057 us; error 41 us; skew 500 ppm
I20260808 03:52:36.780429 9826 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:52:36 dist-test-slave-jlzn krb5kdc[9665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161156, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:52:36.783490 9826 init.cc:377] Logged in from keytab as kudu/127.26.146.129@KRBTEST.COM (short username kudu)
I20260808 03:52:36.784999 9826 webserver.cc:460] Webserver started at http://127.26.146.129:36813/ using document root <none> and password file <none>
I20260808 03:52:36.785698 9826 fs_manager.cc:359] Metadata directory not provided
I20260808 03:52:36.785816 9826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:52:36.786121 9826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:52:36.789019 9826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "14a99013f51d482eb988fc1bb79a26c4"
format_stamp: "Formatted at 2026-08-08 03:52:36 on dist-test-slave-jlzn"
server_key: "d916d537afb996245063a86cba91db1e"
server_key_iv: "9356f3ce681adb4bade70ed9a45e6dde"
server_key_version: "encryptionkey@0"
I20260808 03:52:36.789804 9826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "14a99013f51d482eb988fc1bb79a26c4"
format_stamp: "Formatted at 2026-08-08 03:52:36 on dist-test-slave-jlzn"
server_key: "d916d537afb996245063a86cba91db1e"
server_key_iv: "9356f3ce681adb4bade70ed9a45e6dde"
server_key_version: "encryptionkey@0"
I20260808 03:52:36.794836 9826 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 03:52:36.798247 9843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:36.799552 9826 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:52:36.799706 9826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "14a99013f51d482eb988fc1bb79a26c4"
format_stamp: "Formatted at 2026-08-08 03:52:36 on dist-test-slave-jlzn"
server_key: "d916d537afb996245063a86cba91db1e"
server_key_iv: "9356f3ce681adb4bade70ed9a45e6dde"
server_key_version: "encryptionkey@0"
I20260808 03:52:36.799937 9826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:52:36.817013 9826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:52:36.820883 9826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:52:36.821163 9826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:52:36.822099 9826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:52:36.823516 9826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:52:36.823596 9826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:36.823724 9826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:52:36.823779 9826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:36.853931 9826 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:40731
I20260808 03:52:36.853972 10013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:40731 every 8 connection(s)
I20260808 03:52:36.855161 9826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:52:36.856314 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 9826
I20260808 03:52:36.856541 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:52:36.857350 27210 external_mini_cluster.cc:1428] Setting key f33cff1d8593bc0e7a49824690bbf134
Aug 08 03:52:36 dist-test-slave-jlzn krb5kdc[9665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161156, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:52:36.877722 10018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:36.857895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:49711 (local address 127.26.146.190:42635)
0808 03:52:36.858391 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:36.858397 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:36.859391 (+ 994us) server_negotiation.cc:407] Connection header received
0808 03:52:36.861169 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:36.861175 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:36.861254 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:36.861344 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:36.862787 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:36.863656 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:36.865798 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:36.866708 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:36.866906 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:36.871544 (+ 4638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:36.871549 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:36.871552 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:36.871585 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:36.874048 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:36.874949 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:36.874953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:36.874955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:36.875072 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:36.875559 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:36.875564 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:36.875567 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:36.875958 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:36.876111 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:36.877123 (+ 1012us) server_negotiation.cc:299] Negotiation successful
0808 03:52:36.877350 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":231,"threads_started":1}
I20260808 03:52:36.878963 10019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:36.858458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:42635 (local address 127.26.146.129:49711)
0808 03:52:36.859244 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:36.859301 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:36.860926 (+ 1625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:36.861581 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:36.861589 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:36.862077 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:36.862629 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:36.862645 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:36.863880 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:36.863885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:36.865615 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:36.865631 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:36.866965 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:36.866971 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:36.867260 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:36.868514 (+ 1254us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:36.868549 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:36.871288 (+ 2739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:52:36.874202 (+ 2914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:36.874211 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:36.874228 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:36.874839 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:36.875189 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:36.875193 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:36.875196 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:36.875424 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:36.876127 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:52:36.876169 (+ 42us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:52:36.876652 (+ 483us) client_negotiation.cc:769] Sending connection context
0808 03:52:36.876980 (+ 328us) client_negotiation.cc:240] Negotiation successful
0808 03:52:36.877561 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":307,"threads_started":1}
I20260808 03:52:36.880599 10016 heartbeater.cc:347] Connected to a master server at 127.26.146.190:42635
I20260808 03:52:36.880950 10016 heartbeater.cc:464] Registering TS with master...
I20260808 03:52:36.881839 10016 heartbeater.cc:511] Master 127.26.146.190:42635 requested a full tablet report, sending...
I20260808 03:52:36.884128 9728 ts_manager.cc:194] Registered new tserver with Master: 14a99013f51d482eb988fc1bb79a26c4 (127.26.146.129:40731)
I20260808 03:52:36.885561 9728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.129@KRBTEST.COM'} at 127.26.146.129:49711
WARNING: no policy specified for kudu/127.26.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
I20260808 03:52:36.919157 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:42635
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:46875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:52:37.243631 10027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:37.243954 10027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:37.244064 10027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:37.251041 10027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:37.251142 10027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:37.251256 10027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:52:37.260828 10027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:46875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:42635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:37.262058 10027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:37.263363 10027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:37.275969 10036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:37.275938 10037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:37.276016 10039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:37.276765 10027 server_base.cc:1034] running on GCE node
I20260808 03:52:37.277328 10027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:37.278193 10027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:37.279388 10027 hybrid_clock.cc:648] HybridClock initialized: now 1786161157279339 us; error 45 us; skew 500 ppm
I20260808 03:52:37.279670 10027 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:52:37 dist-test-slave-jlzn krb5kdc[9665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161157, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:52:37.282657 10027 init.cc:377] Logged in from keytab as kudu/127.26.146.130@KRBTEST.COM (short username kudu)
I20260808 03:52:37.284029 10027 webserver.cc:460] Webserver started at http://127.26.146.130:44451/ using document root <none> and password file <none>
I20260808 03:52:37.284623 10027 fs_manager.cc:359] Metadata directory not provided
I20260808 03:52:37.284727 10027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:52:37.284973 10027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:52:37.287650 10027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "0e65730f002344a0b53e4b9202281ee2"
format_stamp: "Formatted at 2026-08-08 03:52:37 on dist-test-slave-jlzn"
server_key: "9f74203bf7dd66554e4223b06281fa9d"
server_key_iv: "13f30b2b92c4db7d67c6d1efbb1f0526"
server_key_version: "encryptionkey@0"
I20260808 03:52:37.288405 10027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "0e65730f002344a0b53e4b9202281ee2"
format_stamp: "Formatted at 2026-08-08 03:52:37 on dist-test-slave-jlzn"
server_key: "9f74203bf7dd66554e4223b06281fa9d"
server_key_iv: "13f30b2b92c4db7d67c6d1efbb1f0526"
server_key_version: "encryptionkey@0"
I20260808 03:52:37.293212 10027 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 03:52:37.296397 10051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:37.297614 10027 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:52:37.297755 10027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "0e65730f002344a0b53e4b9202281ee2"
format_stamp: "Formatted at 2026-08-08 03:52:37 on dist-test-slave-jlzn"
server_key: "9f74203bf7dd66554e4223b06281fa9d"
server_key_iv: "13f30b2b92c4db7d67c6d1efbb1f0526"
server_key_version: "encryptionkey@0"
I20260808 03:52:37.297919 10027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:52:37.313144 10027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:52:37.317107 10027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:52:37.317365 10027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:52:37.318250 10027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:52:37.319499 10027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:52:37.319578 10027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:37.319671 10027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:52:37.319725 10027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:37.349571 10027 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:39319
I20260808 03:52:37.349624 10226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:39319 every 8 connection(s)
I20260808 03:52:37.350848 10027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:52:37.354786 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 10027
I20260808 03:52:37.354987 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:52:37.355551 27210 external_mini_cluster.cc:1428] Setting key b55e0a11ddf74c7f6468099a48abd0b7
Aug 08 03:52:37 dist-test-slave-jlzn krb5kdc[9665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161157, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:52:37.371963 10018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:37.353378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:47523 (local address 127.26.146.190:42635)
0808 03:52:37.353561 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:37.353566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:37.354790 (+ 1224us) server_negotiation.cc:407] Connection header received
0808 03:52:37.357026 (+ 2236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:37.357033 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:37.357109 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:37.357218 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:37.359104 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:37.359854 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:37.361876 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:37.362288 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:37.362382 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:37.366283 (+ 3901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:37.366288 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:37.366291 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:37.366325 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:37.368887 (+ 2562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:37.369695 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:37.369700 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:37.369703 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:37.369864 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:37.370241 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:37.370246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:37.370248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:37.370579 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:37.370726 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:37.371354 (+ 628us) server_negotiation.cc:299] Negotiation successful
0808 03:52:37.371599 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 03:52:37.372783 10232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:37.353825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:42635 (local address 127.26.146.130:47523)
0808 03:52:37.354580 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:37.354650 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:37.356772 (+ 2122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:37.357434 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:37.357446 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:37.358150 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:37.358926 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:37.358951 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:37.360027 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:37.360032 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:37.361759 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:37.361774 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:37.362496 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:37.362502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:37.362784 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:37.363452 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:37.363480 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:37.366055 (+ 2575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:52:37.369034 (+ 2979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:37.369042 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:37.369058 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:37.369585 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:37.369958 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:37.369961 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:37.369964 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:37.370144 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:37.370685 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:52:37.370690 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:52:37.370999 (+ 309us) client_negotiation.cc:769] Sending connection context
0808 03:52:37.371226 (+ 227us) client_negotiation.cc:240] Negotiation successful
0808 03:52:37.371630 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":271,"threads_started":1}
I20260808 03:52:37.374190 10229 heartbeater.cc:347] Connected to a master server at 127.26.146.190:42635
I20260808 03:52:37.374519 10229 heartbeater.cc:464] Registering TS with master...
I20260808 03:52:37.375358 10229 heartbeater.cc:511] Master 127.26.146.190:42635 requested a full tablet report, sending...
I20260808 03:52:37.376952 9728 ts_manager.cc:194] Registered new tserver with Master: 0e65730f002344a0b53e4b9202281ee2 (127.26.146.130:39319)
I20260808 03:52:37.377753 9728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.130@KRBTEST.COM'} at 127.26.146.130:47523
WARNING: no policy specified for kudu/127.26.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
I20260808 03:52:37.417080 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:42635
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:46875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:52:37.737360 10238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:52:37.737653 10238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:52:37.737787 10238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:52:37.744971 10238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:52:37.745075 10238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:52:37.745223 10238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:52:37.754916 10238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:46875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:42635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:52:37.756194 10238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:52:37.757428 10238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:52:37.770131 10244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:37.770174 10246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:52:37.770593 10249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:52:37.770973 10238 server_base.cc:1034] running on GCE node
I20260808 03:52:37.771575 10238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:52:37.772557 10238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:52:37.773770 10238 hybrid_clock.cc:648] HybridClock initialized: now 1786161157773720 us; error 36 us; skew 500 ppm
I20260808 03:52:37.774070 10238 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:52:37 dist-test-slave-jlzn krb5kdc[9665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161157, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:52:37.777266 10238 init.cc:377] Logged in from keytab as kudu/127.26.146.131@KRBTEST.COM (short username kudu)
I20260808 03:52:37.778633 10238 webserver.cc:460] Webserver started at http://127.26.146.131:40019/ using document root <none> and password file <none>
I20260808 03:52:37.779332 10238 fs_manager.cc:359] Metadata directory not provided
I20260808 03:52:37.779465 10238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:52:37.779753 10238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:52:37.782788 10238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "94904cd115c64234b723a8b2182000ce"
format_stamp: "Formatted at 2026-08-08 03:52:37 on dist-test-slave-jlzn"
server_key: "d9d83a25fdce09c8faec5d2878c22027"
server_key_iv: "aca2ba0bba4e7804c45108cb976839b2"
server_key_version: "encryptionkey@0"
I20260808 03:52:37.783552 10238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "94904cd115c64234b723a8b2182000ce"
format_stamp: "Formatted at 2026-08-08 03:52:37 on dist-test-slave-jlzn"
server_key: "d9d83a25fdce09c8faec5d2878c22027"
server_key_iv: "aca2ba0bba4e7804c45108cb976839b2"
server_key_version: "encryptionkey@0"
I20260808 03:52:37.788857 10238 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 03:52:37.792192 10259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:37.793486 10238 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 03:52:37.793656 10238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "94904cd115c64234b723a8b2182000ce"
format_stamp: "Formatted at 2026-08-08 03:52:37 on dist-test-slave-jlzn"
server_key: "d9d83a25fdce09c8faec5d2878c22027"
server_key_iv: "aca2ba0bba4e7804c45108cb976839b2"
server_key_version: "encryptionkey@0"
I20260808 03:52:37.793861 10238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:52:37.806357 10238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:52:37.810292 10238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:52:37.810564 10238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:52:37.811429 10238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:52:37.812759 10238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:52:37.812839 10238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:37.812954 10238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:52:37.813011 10238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:52:37.842768 10238 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:42217
I20260808 03:52:37.842803 10412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:42217 every 8 connection(s)
I20260808 03:52:37.844062 10238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:52:37.851187 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 10238
I20260808 03:52:37.851405 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:52:37.851986 27210 external_mini_cluster.cc:1428] Setting key f3f2100fd7e423e2d0c6770252e80a0d
Aug 08 03:52:37 dist-test-slave-jlzn krb5kdc[9665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161157, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:52:37.864483 10018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:37.846455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:59397 (local address 127.26.146.190:42635)
0808 03:52:37.846622 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:37.846626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:37.847873 (+ 1247us) server_negotiation.cc:407] Connection header received
0808 03:52:37.849682 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:37.849687 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:37.849757 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:37.849845 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:37.851268 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:37.852433 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:37.854683 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:37.855240 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:37.855335 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:37.859064 (+ 3729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:37.859069 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:37.859072 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:37.859104 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:37.861571 (+ 2467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:37.862252 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:37.862256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:37.862258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:37.862389 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:37.862788 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:37.862791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:37.862793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:37.863055 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:37.863186 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:37.863942 (+ 756us) server_negotiation.cc:299] Negotiation successful
0808 03:52:37.864157 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 03:52:37.865327 10417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:37.846949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:42635 (local address 127.26.146.131:59397)
0808 03:52:37.847645 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:37.847727 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:37.849389 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:37.850026 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:37.850034 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:37.850525 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:52:37.851092 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:37.851110 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:37.852643 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:37.852647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:37.854523 (+ 1876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:37.854539 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:37.855439 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:37.855446 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:37.855718 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:37.856411 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:52:37.856440 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:52:37.858871 (+ 2431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:52:37.861714 (+ 2843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:37.861722 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:37.861736 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:37.862141 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:37.862492 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:52:37.862497 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:52:37.862501 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:52:37.862681 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:52:37.863183 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:52:37.863190 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:52:37.863557 (+ 367us) client_negotiation.cc:769] Sending connection context
0808 03:52:37.863786 (+ 229us) client_negotiation.cc:240] Negotiation successful
0808 03:52:37.864296 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":269,"threads_started":1}
I20260808 03:52:37.866602 10415 heartbeater.cc:347] Connected to a master server at 127.26.146.190:42635
I20260808 03:52:37.866920 10415 heartbeater.cc:464] Registering TS with master...
I20260808 03:52:37.867705 10415 heartbeater.cc:511] Master 127.26.146.190:42635 requested a full tablet report, sending...
I20260808 03:52:37.869033 9727 ts_manager.cc:194] Registered new tserver with Master: 94904cd115c64234b723a8b2182000ce (127.26.146.131:42217)
I20260808 03:52:37.869868 9727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.131@KRBTEST.COM'} at 127.26.146.131:59397
I20260808 03:52:37.880403 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:52:37.888456 10016 heartbeater.cc:503] Master 127.26.146.190:42635 was elected leader, sending a full tablet report...
I20260808 03:52:37.896338 10018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:37.884860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41832 (local address 127.26.146.190:42635)
0808 03:52:37.885012 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:37.885016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:37.885570 (+ 554us) server_negotiation.cc:407] Connection header received
0808 03:52:37.885630 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:37.885633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:37.885705 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:37.885791 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:37.887123 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:37.887912 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:37.889165 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:37.889598 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:37.889709 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:37.891849 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:37.891857 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:37.891862 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:37.891897 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:37.894288 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:37.894811 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:37.894815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:37.894817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:37.894943 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:37.895205 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:37.895208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:37.895210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:37.895503 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:37.895690 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:37.895948 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 03:52:37.896109 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 03:52:37.900288 9727 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:52:37.907691 9727 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:52:37.944541 10429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:37.919737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:39319 (local address 127.0.0.1:52946)
0808 03:52:37.934898 (+ 15161us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:37.934966 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:37.935169 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:37.937750 (+ 2581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:37.937758 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:37.937788 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:52:37.938224 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:37.938239 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:37.940090 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:37.940095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:37.941591 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:37.941603 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:37.943464 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:37.943470 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:37.943700 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:37.943767 (+ 67us) client_negotiation.cc:769] Sending connection context
0808 03:52:37.943929 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 03:52:37.944117 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14848,"mutex_wait_us":3,"thread_start_us":15640,"threads_started":1}
I20260808 03:52:37.945859 10430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:37.919982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52946 (local address 127.26.146.130:39319)
0808 03:52:37.937039 (+ 17057us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:37.937048 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:37.937078 (+ 30us) server_negotiation.cc:407] Connection header received
0808 03:52:37.937185 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:37.937193 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:37.937470 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:37.937733 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:52:37.938441 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:37.939997 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:37.941774 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:37.943222 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:37.943421 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:37.943595 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:37.945380 (+ 1785us) server_negotiation.cc:299] Negotiation successful
0808 03:52:37.945538 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16917,"thread_start_us":16791,"threads_started":1}
I20260808 03:52:37.946069 10432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:37.920272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42922 (local address 127.26.146.129:40731)
0808 03:52:37.936617 (+ 16345us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:37.936628 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:37.937884 (+ 1256us) server_negotiation.cc:407] Connection header received
0808 03:52:37.938786 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:37.938795 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:37.939013 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:37.939191 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:52:37.939869 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:37.941385 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:37.943966 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:37.945028 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:37.945184 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:37.945294 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:37.945595 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 03:52:37.945757 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16145,"thread_start_us":15984,"threads_started":1}
I20260808 03:52:37.946584 10433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:37.921251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:42217 (local address 127.0.0.1:42764)
0808 03:52:37.938716 (+ 17465us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:37.938732 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:37.938836 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:37.939482 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:37.939487 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:37.939515 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:52:37.939914 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:37.939926 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:37.941733 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:37.941738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:37.943922 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:37.943937 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:37.945585 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:37.945591 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:37.945756 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:37.945785 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 03:52:37.945873 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 03:52:37.946270 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17347,"mutex_wait_us":3,"thread_start_us":16215,"threads_started":1}
I20260808 03:52:37.946813 10434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:37.920725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:40731 (local address 127.0.0.1:42922)
0808 03:52:37.937752 (+ 17027us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:37.937772 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:37.938515 (+ 743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:37.939199 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:37.939204 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:37.939226 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:52:37.939655 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:37.939667 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:37.941526 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:37.941532 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:37.943824 (+ 2292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:37.943836 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:37.945229 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:37.945236 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:37.945388 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:37.945420 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 03:52:37.946278 (+ 858us) client_negotiation.cc:240] Negotiation successful
0808 03:52:37.946403 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16876,"mutex_wait_us":68,"thread_start_us":17711,"threads_started":1}
I20260808 03:52:37.947305 10435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:37.921778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42764 (local address 127.26.146.131:42217)
0808 03:52:37.938393 (+ 16615us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:37.938401 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:37.938833 (+ 432us) server_negotiation.cc:407] Connection header received
0808 03:52:37.938968 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:37.938973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:37.939142 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:37.939330 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:52:37.940107 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:37.941604 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:37.944081 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:37.945373 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:37.945560 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:37.945724 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:37.945939 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 03:52:37.947062 (+ 1123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16458,"thread_start_us":16309,"threads_started":1}
I20260808 03:52:37.949537 10132 tablet_service.cc:1467] Processing CreateTablet for tablet 7e0adcaecfa54083afbabcadc440716e (DEFAULT_TABLE table=test-table [id=5cfb0dc26bc94cee8129e3e1fe398c81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:52:37.950191 10320 tablet_service.cc:1467] Processing CreateTablet for tablet 7e0adcaecfa54083afbabcadc440716e (DEFAULT_TABLE table=test-table [id=5cfb0dc26bc94cee8129e3e1fe398c81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:52:37.950945 10132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e0adcaecfa54083afbabcadc440716e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:52:37.951361 10320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e0adcaecfa54083afbabcadc440716e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:52:37.960898 9917 tablet_service.cc:1467] Processing CreateTablet for tablet 7e0adcaecfa54083afbabcadc440716e (DEFAULT_TABLE table=test-table [id=5cfb0dc26bc94cee8129e3e1fe398c81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:52:37.961504 10438 tablet_bootstrap.cc:492] T 7e0adcaecfa54083afbabcadc440716e P 0e65730f002344a0b53e4b9202281ee2: Bootstrap starting.
I20260808 03:52:37.962366 9917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e0adcaecfa54083afbabcadc440716e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:52:37.962405 10439 tablet_bootstrap.cc:492] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce: Bootstrap starting.
I20260808 03:52:37.965102 10438 tablet_bootstrap.cc:654] T 7e0adcaecfa54083afbabcadc440716e P 0e65730f002344a0b53e4b9202281ee2: Neither blocks nor log segments found. Creating new log.
I20260808 03:52:37.965821 10439 tablet_bootstrap.cc:654] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce: Neither blocks nor log segments found. Creating new log.
I20260808 03:52:37.966358 10438 log.cc:824] T 7e0adcaecfa54083afbabcadc440716e P 0e65730f002344a0b53e4b9202281ee2: Log is configured to *not* fsync() on all Append() calls
I20260808 03:52:37.967144 10439 log.cc:824] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce: Log is configured to *not* fsync() on all Append() calls
I20260808 03:52:37.969552 10438 tablet_bootstrap.cc:492] T 7e0adcaecfa54083afbabcadc440716e P 0e65730f002344a0b53e4b9202281ee2: No bootstrap required, opened a new log
I20260808 03:52:37.969887 10438 ts_tablet_manager.cc:1397] T 7e0adcaecfa54083afbabcadc440716e P 0e65730f002344a0b53e4b9202281ee2: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20260808 03:52:37.970220 10439 tablet_bootstrap.cc:492] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce: No bootstrap required, opened a new log
I20260808 03:52:37.970443 10439 ts_tablet_manager.cc:1397] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260808 03:52:37.976177 10442 tablet_bootstrap.cc:492] T 7e0adcaecfa54083afbabcadc440716e P 14a99013f51d482eb988fc1bb79a26c4: Bootstrap starting.
I20260808 03:52:37.977617 10439 raft_consensus.cc:348] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e65730f002344a0b53e4b9202281ee2" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 39319 } } peers { permanent_uuid: "14a99013f51d482eb988fc1bb79a26c4" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40731 } } peers { permanent_uuid: "94904cd115c64234b723a8b2182000ce" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 42217 } }
I20260808 03:52:37.978070 10439 raft_consensus.cc:374] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:52:37.978173 10439 raft_consensus.cc:729] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94904cd115c64234b723a8b2182000ce, State: Initialized, Role: FOLLOWER
I20260808 03:52:37.978770 10439 consensus_queue.cc:260] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e65730f002344a0b53e4b9202281ee2" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 39319 } } peers { permanent_uuid: "14a99013f51d482eb988fc1bb79a26c4" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40731 } } peers { permanent_uuid: "94904cd115c64234b723a8b2182000ce" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 42217 } }
I20260808 03:52:37.978955 10438 raft_consensus.cc:348] T 7e0adcaecfa54083afbabcadc440716e P 0e65730f002344a0b53e4b9202281ee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e65730f002344a0b53e4b9202281ee2" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 39319 } } peers { permanent_uuid: "14a99013f51d482eb988fc1bb79a26c4" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40731 } } peers { permanent_uuid: "94904cd115c64234b723a8b2182000ce" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 42217 } }
I20260808 03:52:37.979321 10438 raft_consensus.cc:374] T 7e0adcaecfa54083afbabcadc440716e P 0e65730f002344a0b53e4b9202281ee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:52:37.979426 10438 raft_consensus.cc:729] T 7e0adcaecfa54083afbabcadc440716e P 0e65730f002344a0b53e4b9202281ee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e65730f002344a0b53e4b9202281ee2, State: Initialized, Role: FOLLOWER
I20260808 03:52:37.980036 10415 heartbeater.cc:503] Master 127.26.146.190:42635 was elected leader, sending a full tablet report...
I20260808 03:52:37.980002 10438 consensus_queue.cc:260] T 7e0adcaecfa54083afbabcadc440716e P 0e65730f002344a0b53e4b9202281ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e65730f002344a0b53e4b9202281ee2" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 39319 } } peers { permanent_uuid: "14a99013f51d482eb988fc1bb79a26c4" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40731 } } peers { permanent_uuid: "94904cd115c64234b723a8b2182000ce" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 42217 } }
I20260808 03:52:37.980643 10439 ts_tablet_manager.cc:1428] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260808 03:52:37.981066 10229 heartbeater.cc:503] Master 127.26.146.190:42635 was elected leader, sending a full tablet report...
I20260808 03:52:37.981321 10442 tablet_bootstrap.cc:654] T 7e0adcaecfa54083afbabcadc440716e P 14a99013f51d482eb988fc1bb79a26c4: Neither blocks nor log segments found. Creating new log.
I20260808 03:52:37.981577 10438 ts_tablet_manager.cc:1428] T 7e0adcaecfa54083afbabcadc440716e P 0e65730f002344a0b53e4b9202281ee2: Time spent starting tablet: real 0.012s user 0.009s sys 0.002s
I20260808 03:52:37.982640 10442 log.cc:824] T 7e0adcaecfa54083afbabcadc440716e P 14a99013f51d482eb988fc1bb79a26c4: Log is configured to *not* fsync() on all Append() calls
I20260808 03:52:37.986096 10442 tablet_bootstrap.cc:492] T 7e0adcaecfa54083afbabcadc440716e P 14a99013f51d482eb988fc1bb79a26c4: No bootstrap required, opened a new log
I20260808 03:52:37.986375 10442 ts_tablet_manager.cc:1397] T 7e0adcaecfa54083afbabcadc440716e P 14a99013f51d482eb988fc1bb79a26c4: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260808 03:52:37.993194 10442 raft_consensus.cc:348] T 7e0adcaecfa54083afbabcadc440716e P 14a99013f51d482eb988fc1bb79a26c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e65730f002344a0b53e4b9202281ee2" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 39319 } } peers { permanent_uuid: "14a99013f51d482eb988fc1bb79a26c4" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40731 } } peers { permanent_uuid: "94904cd115c64234b723a8b2182000ce" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 42217 } }
I20260808 03:52:37.993499 10442 raft_consensus.cc:374] T 7e0adcaecfa54083afbabcadc440716e P 14a99013f51d482eb988fc1bb79a26c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:52:37.993577 10442 raft_consensus.cc:729] T 7e0adcaecfa54083afbabcadc440716e P 14a99013f51d482eb988fc1bb79a26c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14a99013f51d482eb988fc1bb79a26c4, State: Initialized, Role: FOLLOWER
I20260808 03:52:37.994014 10442 consensus_queue.cc:260] T 7e0adcaecfa54083afbabcadc440716e P 14a99013f51d482eb988fc1bb79a26c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e65730f002344a0b53e4b9202281ee2" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 39319 } } peers { permanent_uuid: "14a99013f51d482eb988fc1bb79a26c4" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40731 } } peers { permanent_uuid: "94904cd115c64234b723a8b2182000ce" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 42217 } }
I20260808 03:52:37.995362 10442 ts_tablet_manager.cc:1428] T 7e0adcaecfa54083afbabcadc440716e P 14a99013f51d482eb988fc1bb79a26c4: Time spent starting tablet: real 0.009s user 0.006s sys 0.004s
I20260808 03:52:38.044296 10443 raft_consensus.cc:482] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:52:38.044595 10443 raft_consensus.cc:504] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e65730f002344a0b53e4b9202281ee2" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 39319 } } peers { permanent_uuid: "14a99013f51d482eb988fc1bb79a26c4" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40731 } } peers { permanent_uuid: "94904cd115c64234b723a8b2182000ce" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 42217 } }
I20260808 03:52:38.046307 10443 leader_election.cc:288] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e65730f002344a0b53e4b9202281ee2 (127.26.146.130:39319), 14a99013f51d482eb988fc1bb79a26c4 (127.26.146.129:40731)
I20260808 03:52:38.052301 10432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:38.046876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:60845 (local address 127.26.146.129:40731)
0808 03:52:38.047118 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:38.047125 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:38.047182 (+ 57us) server_negotiation.cc:407] Connection header received
0808 03:52:38.047418 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:38.047422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:38.047484 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:38.047564 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:52:38.047978 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:38.048712 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:38.050459 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:38.051452 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:38.051578 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:38.051631 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:38.051970 (+ 339us) server_negotiation.cc:299] Negotiation successful
0808 03:52:38.052059 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 03:52:38.052362 10417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:38.046726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:40731 (local address 127.26.146.131:60845)
0808 03:52:38.047076 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:38.047111 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:38.047268 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:38.047612 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:38.047617 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:38.047644 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:52:38.047826 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:38.047835 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:38.048840 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:38.048845 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:38.050295 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:38.050308 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:38.051604 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:38.051611 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:38.051795 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:38.051855 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 03:52:38.051950 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 03:52:38.052084 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217}
I20260808 03:52:38.052906 10430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:38.046798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:39059 (local address 127.26.146.130:39319)
0808 03:52:38.047002 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:38.047009 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:38.047568 (+ 559us) server_negotiation.cc:407] Connection header received
0808 03:52:38.047707 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:38.047711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:38.047769 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:38.047879 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:52:38.048390 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:38.049255 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:38.051080 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:38.052067 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:38.052208 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:38.052275 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:38.052627 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 03:52:38.052733 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 03:52:38.053373 10448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:38.046766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:39319 (local address 127.26.146.131:39059)
0808 03:52:38.047459 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0808 03:52:38.047472 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 03:52:38.047580 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:52:38.047903 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:52:38.047907 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:52:38.047930 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:52:38.048241 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:38.048249 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:38.049396 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:38.049400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:38.050923 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:52:38.050937 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:52:38.052264 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:52:38.052270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:52:38.052436 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:38.052467 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 03:52:38.053058 (+ 591us) client_negotiation.cc:240] Negotiation successful
0808 03:52:38.053154 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":409,"threads_started":1}
I20260808 03:52:38.053321 9946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e0adcaecfa54083afbabcadc440716e" candidate_uuid: "94904cd115c64234b723a8b2182000ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14a99013f51d482eb988fc1bb79a26c4" is_pre_election: true
I20260808 03:52:38.053869 9946 raft_consensus.cc:2454] T 7e0adcaecfa54083afbabcadc440716e P 14a99013f51d482eb988fc1bb79a26c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94904cd115c64234b723a8b2182000ce in term 0.
I20260808 03:52:38.054253 10163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e0adcaecfa54083afbabcadc440716e" candidate_uuid: "94904cd115c64234b723a8b2182000ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e65730f002344a0b53e4b9202281ee2" is_pre_election: true
I20260808 03:52:38.054491 10260 leader_election.cc:302] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14a99013f51d482eb988fc1bb79a26c4, 94904cd115c64234b723a8b2182000ce; no voters:
I20260808 03:52:38.054661 10163 raft_consensus.cc:2454] T 7e0adcaecfa54083afbabcadc440716e P 0e65730f002344a0b53e4b9202281ee2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94904cd115c64234b723a8b2182000ce in term 0.
I20260808 03:52:38.054857 10443 raft_consensus.cc:2781] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:52:38.054977 10443 raft_consensus.cc:482] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:52:38.055086 10443 raft_consensus.cc:3037] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:52:38.057284 10443 raft_consensus.cc:504] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e65730f002344a0b53e4b9202281ee2" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 39319 } } peers { permanent_uuid: "14a99013f51d482eb988fc1bb79a26c4" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40731 } } peers { permanent_uuid: "94904cd115c64234b723a8b2182000ce" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 42217 } }
I20260808 03:52:38.057816 10443 leader_election.cc:288] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce [CANDIDATE]: Term 1 election: Requested vote from peers 0e65730f002344a0b53e4b9202281ee2 (127.26.146.130:39319), 14a99013f51d482eb988fc1bb79a26c4 (127.26.146.129:40731)
I20260808 03:52:38.058192 10163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e0adcaecfa54083afbabcadc440716e" candidate_uuid: "94904cd115c64234b723a8b2182000ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e65730f002344a0b53e4b9202281ee2"
I20260808 03:52:38.058254 9946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e0adcaecfa54083afbabcadc440716e" candidate_uuid: "94904cd115c64234b723a8b2182000ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14a99013f51d482eb988fc1bb79a26c4"
I20260808 03:52:38.058359 10163 raft_consensus.cc:3037] T 7e0adcaecfa54083afbabcadc440716e P 0e65730f002344a0b53e4b9202281ee2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:52:38.058439 9946 raft_consensus.cc:3037] T 7e0adcaecfa54083afbabcadc440716e P 14a99013f51d482eb988fc1bb79a26c4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:52:38.060606 9946 raft_consensus.cc:2454] T 7e0adcaecfa54083afbabcadc440716e P 14a99013f51d482eb988fc1bb79a26c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94904cd115c64234b723a8b2182000ce in term 1.
I20260808 03:52:38.060752 10163 raft_consensus.cc:2454] T 7e0adcaecfa54083afbabcadc440716e P 0e65730f002344a0b53e4b9202281ee2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94904cd115c64234b723a8b2182000ce in term 1.
I20260808 03:52:38.061003 10260 leader_election.cc:302] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14a99013f51d482eb988fc1bb79a26c4, 94904cd115c64234b723a8b2182000ce; no voters:
I20260808 03:52:38.061321 10443 raft_consensus.cc:2781] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:52:38.061717 10443 raft_consensus.cc:686] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce [term 1 LEADER]: Becoming Leader. State: Replica: 94904cd115c64234b723a8b2182000ce, State: Running, Role: LEADER
I20260808 03:52:38.062139 10443 consensus_queue.cc:237] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e65730f002344a0b53e4b9202281ee2" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 39319 } } peers { permanent_uuid: "14a99013f51d482eb988fc1bb79a26c4" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40731 } } peers { permanent_uuid: "94904cd115c64234b723a8b2182000ce" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 42217 } }
I20260808 03:52:38.066699 9728 catalog_manager.cc:5495] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 94904cd115c64234b723a8b2182000ce (127.26.146.131). New cstate: current_term: 1 leader_uuid: "94904cd115c64234b723a8b2182000ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e65730f002344a0b53e4b9202281ee2" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 39319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14a99013f51d482eb988fc1bb79a26c4" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94904cd115c64234b723a8b2182000ce" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 42217 } health_report { overall_health: HEALTHY } } }
I20260808 03:52:38.083535 10435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:38.078449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42778 (local address 127.26.146.131:42217)
0808 03:52:38.078653 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:38.078658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:38.078677 (+ 19us) server_negotiation.cc:407] Connection header received
0808 03:52:38.078851 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:38.078854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:38.078926 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:38.079049 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:52:38.079431 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:38.080179 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:38.081466 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:38.081893 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:38.082023 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:38.082495 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:52:38.082988 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:52:38.083120 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:38.083229 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 03:52:38.083306 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 03:52:38.090907 9946 raft_consensus.cc:1261] T 7e0adcaecfa54083afbabcadc440716e P 14a99013f51d482eb988fc1bb79a26c4 [term 1 FOLLOWER]: Refusing update from remote peer 94904cd115c64234b723a8b2182000ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:52:38.090909 10163 raft_consensus.cc:1261] T 7e0adcaecfa54083afbabcadc440716e P 0e65730f002344a0b53e4b9202281ee2 [term 1 FOLLOWER]: Refusing update from remote peer 94904cd115c64234b723a8b2182000ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:52:38.091840 10450 consensus_queue.cc:1035] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e65730f002344a0b53e4b9202281ee2" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 39319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:52:38.092161 10443 consensus_queue.cc:1035] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "14a99013f51d482eb988fc1bb79a26c4" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 03:52:38.095505 10416 tablet.cc:2369] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:52:38.102671 10230 tablet.cc:2369] T 7e0adcaecfa54083afbabcadc440716e P 0e65730f002344a0b53e4b9202281ee2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:52:38.105495 10454 mvcc.cc:205] Tried to move back new op lower bound from 7316116103531524096 to 7316116103426265088. Current Snapshot: MvccSnapshot[applied={T|T < 7316116103531524096}]
I20260808 03:52:38.122413 9728 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:52:38.122674 9728 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:52:38.126168 9728 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d46ace43fb91402881819090fef1355d: Sending DeleteTablet for 3 replicas of tablet 7e0adcaecfa54083afbabcadc440716e
I20260808 03:52:38.127347 10132 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e0adcaecfa54083afbabcadc440716e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:52:38 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:52946
I20260808 03:52:38.127429 9917 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e0adcaecfa54083afbabcadc440716e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:52:38 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:42922
I20260808 03:52:38.127786 10320 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e0adcaecfa54083afbabcadc440716e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:52:38 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:42764
I20260808 03:52:38.129901 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 9826
I20260808 03:52:38.140255 10466 tablet_replica.cc:330] T 7e0adcaecfa54083afbabcadc440716e P 0e65730f002344a0b53e4b9202281ee2: stopping tablet replica
I20260808 03:52:38.140728 10466 raft_consensus.cc:2229] T 7e0adcaecfa54083afbabcadc440716e P 0e65730f002344a0b53e4b9202281ee2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:52:38.141278 10466 raft_consensus.cc:2258] T 7e0adcaecfa54083afbabcadc440716e P 0e65730f002344a0b53e4b9202281ee2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:52:38.144269 10467 tablet_replica.cc:330] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce: stopping tablet replica
I20260808 03:52:38.144500 10465 tablet_replica.cc:330] T 7e0adcaecfa54083afbabcadc440716e P 14a99013f51d482eb988fc1bb79a26c4: stopping tablet replica
I20260808 03:52:38.144697 10467 raft_consensus.cc:2229] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:52:38.144891 10465 raft_consensus.cc:2229] T 7e0adcaecfa54083afbabcadc440716e P 14a99013f51d482eb988fc1bb79a26c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:52:38.145408 10465 raft_consensus.cc:2258] T 7e0adcaecfa54083afbabcadc440716e P 14a99013f51d482eb988fc1bb79a26c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:52:38.145386 10467 raft_consensus.cc:2258] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:52:38.146838 10466 ts_tablet_manager.cc:1905] T 7e0adcaecfa54083afbabcadc440716e P 0e65730f002344a0b53e4b9202281ee2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:52:38.147361 10465 ts_tablet_manager.cc:1905] T 7e0adcaecfa54083afbabcadc440716e P 14a99013f51d482eb988fc1bb79a26c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:52:38.147356 10467 ts_tablet_manager.cc:1905] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:52:38.153554 10466 ts_tablet_manager.cc:1918] T 7e0adcaecfa54083afbabcadc440716e P 0e65730f002344a0b53e4b9202281ee2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:52:38.153733 10466 log.cc:1196] T 7e0adcaecfa54083afbabcadc440716e P 0e65730f002344a0b53e4b9202281ee2: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/7e0adcaecfa54083afbabcadc440716e
I20260808 03:52:38.154003 10465 ts_tablet_manager.cc:1918] T 7e0adcaecfa54083afbabcadc440716e P 14a99013f51d482eb988fc1bb79a26c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:52:38.154173 10465 log.cc:1196] T 7e0adcaecfa54083afbabcadc440716e P 14a99013f51d482eb988fc1bb79a26c4: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-0/wal/wals/7e0adcaecfa54083afbabcadc440716e
I20260808 03:52:38.154268 10466 ts_tablet_manager.cc:1939] T 7e0adcaecfa54083afbabcadc440716e P 0e65730f002344a0b53e4b9202281ee2: Deleting consensus metadata
I20260808 03:52:38.154282 10467 ts_tablet_manager.cc:1918] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:52:38.154433 10467 log.cc:1196] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/7e0adcaecfa54083afbabcadc440716e
I20260808 03:52:38.154721 10465 ts_tablet_manager.cc:1939] T 7e0adcaecfa54083afbabcadc440716e P 14a99013f51d482eb988fc1bb79a26c4: Deleting consensus metadata
I20260808 03:52:38.154917 10467 ts_tablet_manager.cc:1939] T 7e0adcaecfa54083afbabcadc440716e P 94904cd115c64234b723a8b2182000ce: Deleting consensus metadata
Aug 08 03:52:38 dist-test-slave-jlzn krb5kdc[9665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.129@KRBTEST.COM
I20260808 03:52:38.156167 9706 catalog_manager.cc:4841] TS 0e65730f002344a0b53e4b9202281ee2 (127.26.146.130:39319): tablet 7e0adcaecfa54083afbabcadc440716e (table test-table [id=5cfb0dc26bc94cee8129e3e1fe398c81]) successfully deleted
I20260808 03:52:38.156395 9703 catalog_manager.cc:4841] TS 94904cd115c64234b723a8b2182000ce (127.26.146.131:42217): tablet 7e0adcaecfa54083afbabcadc440716e (table test-table [id=5cfb0dc26bc94cee8129e3e1fe398c81]) successfully deleted
I20260808 03:52:38.156586 9702 catalog_manager.cc:4841] TS 14a99013f51d482eb988fc1bb79a26c4 (127.26.146.129:40731): tablet 7e0adcaecfa54083afbabcadc440716e (table test-table [id=5cfb0dc26bc94cee8129e3e1fe398c81]) successfully deleted
I20260808 03:52:38.163400 10432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:38.131557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42924 (local address 127.26.146.129:40731)
0808 03:52:38.131745 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:38.131752 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:38.144781 (+ 13029us) server_negotiation.cc:407] Connection header received
0808 03:52:38.144992 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:38.144998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:38.145088 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:38.145186 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:38.147174 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:38.148288 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:38.150129 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:38.150759 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:38.150861 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:38.157131 (+ 6270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:38.157145 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:38.157163 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:38.157204 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:38.160730 (+ 3526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:38.161355 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:38.161364 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:38.161368 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:38.161524 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:38.161863 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:38.161868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:38.161871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:38.162346 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:38.162666 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:38.162813 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 03:52:38.163029 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 03:52:38.164234 10007 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:38.252170 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 9826
I20260808 03:52:38.276906 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 10027
Aug 08 03:52:38 dist-test-slave-jlzn krb5kdc[9665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.130@KRBTEST.COM
I20260808 03:52:38.291862 10430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:38.277666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52958 (local address 127.26.146.130:39319)
0808 03:52:38.278145 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:38.278150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:38.278165 (+ 15us) server_negotiation.cc:407] Connection header received
0808 03:52:38.278226 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:38.278230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:38.278290 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:38.278373 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:38.279620 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:38.280388 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:38.281583 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:38.281960 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:38.282069 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:38.286394 (+ 4325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:38.286410 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:38.286428 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:38.286464 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:38.289132 (+ 2668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:38.289901 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:38.289907 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:38.289909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:38.290024 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:38.290384 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:38.290387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:38.290389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:38.290733 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:38.290952 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:38.291276 (+ 324us) server_negotiation.cc:299] Negotiation successful
0808 03:52:38.291479 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370}
I20260808 03:52:38.292486 10221 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:38.383625 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 10027
I20260808 03:52:38.406980 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 10238
Aug 08 03:52:38 dist-test-slave-jlzn krb5kdc[9665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.131@KRBTEST.COM
I20260808 03:52:38.427861 10435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:52:38.407857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42794 (local address 127.26.146.131:42217)
0808 03:52:38.408038 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 03:52:38.408042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:52:38.408106 (+ 64us) server_negotiation.cc:407] Connection header received
0808 03:52:38.408301 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:52:38.408305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:52:38.408379 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:52:38.408456 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:52:38.410243 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:38.411303 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:38.413331 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:52:38.413819 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:52:38.413936 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:52:38.421953 (+ 8017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:52:38.421965 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:52:38.421981 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:52:38.422020 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:52:38.425042 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:38.425835 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:38.425842 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:38.425844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:38.425994 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:52:38.426410 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:52:38.426413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:52:38.426415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:52:38.426767 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:52:38.427014 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0808 03:52:38.427346 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 03:52:38.427507 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 03:52:38.428592 10408 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:38.548177 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 10238
I20260808 03:52:38.574038 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 9682
I20260808 03:52:38.575271 9781 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:52:38.683759 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 9682
2026-08-08T03:52:38Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4314 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:52:38 dist-test-slave-jlzn krb5kdc[10479](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:52:38 dist-test-slave-jlzn krb5kdc[10479](info): set up 2 sockets
Aug 08 03:52:38 dist-test-slave-jlzn krb5kdc[10479](info): commencing operation
krb5kdc: starting...
W20260808 03:52:39.898747 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.004s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:52:40 dist-test-slave-jlzn krb5kdc[10479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:52:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:52:40Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.26.146.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.26.146.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.26.146.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/rangeradmin_127.26.146.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.26.146.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/rangeradmin_127.26.146.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.26.146.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.26.146.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.26.146.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/rangerlookup_127.26.146.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.26.146.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/rangerlookup_127.26.146.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.146.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.146.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.146.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/HTTP_127.26.146.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.146.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/HTTP_127.26.146.148@KRBTEST.COM.keytab.
I20260808 03:52:40.273058 27210 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskT_CS6X/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/postgres -l logfile start
2026-08-08 03:52:42.586 UTC [10528] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 03:52:42.586 UTC [10528] LOG: listening on IPv4 address "127.26.146.148", port 58873
2026-08-08 03:52:42.589 UTC [10528] LOG: listening on Unix socket "/tmp/.s.PGSQL.58873"
2026-08-08 03:52:42.611 UTC [10534] LOG: database system was shut down at 2026-08-08 03:52:41 UTC
2026-08-08 03:52:42.614 UTC [10528] LOG: database system is ready to accept connections
I20260808 03:52:43.666863 27210 mini_postgres.cc:96] Postgres bound to 58873
2026-08-08 03:52:43.691 UTC [10547] FATAL: database "slave" does not exist
127.26.146.148:58873 - accepting connections
I20260808 03:52:43.692148 27210 mini_ranger.cc:161] Starting Ranger...
I20260808 03:52:43.722312 27210 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 03:52:44.156039 27210 mini_ranger.cc:87] Created ranger Postgres database
I20260808 03:52:44.156183 27210 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ranger-admin
2026-08-08 03:52:44,495 [I] DB FLAVOR :POSTGRES
2026-08-08 03:52:44,495 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 03:52:44,495 [I] Checking connection..
2026-08-08 03:52:44,495 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:58873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 03:52:44,759 [I] Checking connection passed.
2026-08-08 03:52:44,760 [I] --------- Verifying version history table ---------
2026-08-08 03:52:44,760 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:58873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 03:52:45,006 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 03:52:45,006 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:58873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 03:52:45,263 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 03:52:45,264 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 03:52:45,264 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:58873/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 03:52:45.514 UTC [10638] WARNING: there is no transaction in progress
2026-08-08 03:52:45,523 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 03:52:45,523 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:58873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 03:52:45,771 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 03:52:45,772 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 03:52:45,772 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:58873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 03:52:46,033 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:58873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 03:52:46,280 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:58873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-jlzn.c.gcp-upstream.internal','N') ;"
2026-08-08 03:52:46,522 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:58873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 03:52:46,759 [I] Table x_portal_user does not exist in database ranger
2026-08-08 03:52:46,759 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:58873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 03:52:47,021 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 03:52:47,021 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:58873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 03:52:47,263 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:58873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 03:52:47,518 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 03:52:47,518 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:58873/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 03:52:48.424 UTC [10810] WARNING: there is no transaction in progress
2026-08-08 03:52:48.440 UTC [10810] WARNING: there is no transaction in progress
2026-08-08 03:52:48.455 UTC [10810] WARNING: there is no transaction in progress
2026-08-08 03:52:48.469 UTC [10810] WARNING: there is no transaction in progress
2026-08-08 03:52:48.484 UTC [10810] WARNING: there is no transaction in progress
2026-08-08 03:52:48.501 UTC [10810] WARNING: there is no transaction in progress
2026-08-08 03:52:48.564 UTC [10810] WARNING: there is no transaction in progress
2026-08-08 03:52:48.571 UTC [10810] WARNING: there is no transaction in progress
2026-08-08 03:52:48.584 UTC [10810] WARNING: there is no transaction in progress
2026-08-08 03:52:48.593 UTC [10810] WARNING: there is no transaction in progress
2026-08-08 03:52:48.602 UTC [10810] WARNING: there is no transaction in progress
2026-08-08 03:52:48.614 UTC [10810] WARNING: there is no transaction in progress
2026-08-08 03:52:48.624 UTC [10810] WARNING: there is no transaction in progress
2026-08-08 03:52:48.630 UTC [10810] WARNING: there is no transaction in progress
2026-08-08 03:52:48.636 UTC [10810] WARNING: there is no transaction in progress
2026-08-08 03:52:48.642 UTC [10810] WARNING: there is no transaction in progress
2026-08-08 03:52:49,232 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 03:52:49,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:58873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 03:52:49,471 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 03:52:49,472 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:58873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 03:52:49,731 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 03:52:49,731 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:58873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 03:52:49,973 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 03:52:49,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:58873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 03:52:50,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:58873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 03:52:50,475 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:58873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-jlzn.c.gcp-upstream.internal';"
2026-08-08 03:52:50,729 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 03:52:50,730 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:58873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 03:52:50,969 [I] DB_PATCHES have already been applied
I20260808 03:52:50.975626 27210 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ranger-admin:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskT_CS6X/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 03:52:50.975728 27210 mini_ranger.cc:193] Using host: 127.26.146.148
I20260808 03:52:50.996620 27210 mini_ranger.cc:227] Ranger admin URL: http://127.26.146.148:48957
Aug 08, 2026 3:52:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 3:52:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 3:52:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 3:52:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 3:52:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 3:52:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 3:52:51 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 03:52:52.237697 27210 mini_ranger.cc:160] Time spent starting Ranger: real 8.546s user 0.000s sys 0.062s
Aug 08, 2026 3:53:11 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 3:53:11 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 08, 2026 3:53:11 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 3:53:11 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 3:53:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 3:53:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 3:53:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 3:53:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 3:53:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 3:53:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 3:53:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 3:53:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 3:53:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 3:53:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 3:53:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 3:53:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 3:53:11 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 3:53:12 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 03:53:12.724637 27210 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.26.146.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.26.146.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.26.146.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/rangerkms_127.26.146.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.26.146.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/rangerkms_127.26.146.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.146.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.146.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.146.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/HTTP_127.26.146.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.146.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/HTTP_127.26.146.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 03:53:12 dist-test-slave-jlzn krb5kdc[10479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161192, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 03:53:12.932117 27210 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 03:53:12.960489 27210 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 03:53:13.777983 27210 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 03:53:13.778172 27210 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ranger-kms
I20260808 03:53:13.778308 27210 mini_ranger_kms.cc:227] Using postgres at 127.26.146.148:58873
2026-08-08 03:53:14,500 [I] DB FLAVOR :POSTGRES
2026-08-08 03:53:14,501 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 03:53:14,501 [I] Checking connection
2026-08-08 03:53:14,501 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:58873/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 03:53:14,735 [I] connection success
2026-08-08 03:53:14,735 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 03:53:14,735 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 03:53:14,735 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:58873/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 03:53:14,972 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 03:53:14,973 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 03:53:14,973 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 03:53:14,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:58873/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 03:53:15,254 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 03:53:15.714607 27210 mini_ranger_kms.cc:310] Created kms service
I20260808 03:53:15.906418 27210 mini_ranger_kms.cc:326] Created kudu user
I20260808 03:53:15.948385 27210 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 03:53:15 dist-test-slave-jlzn krb5kdc[10479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161192, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.26.146.148@KRBTEST.COM
I20260808 03:53:16.332825 27210 mini_ranger_kms.cc:384] Added ranger policy
I20260808 03:53:16.333185 27210 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ranger-kms:/tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 03:53:16.333243 27210 mini_ranger_kms.cc:240] Using host: 127.26.146.148
I20260808 03:53:16.356990 27210 mini_ranger_kms.cc:276] Ranger KMS PID: 11062
I20260808 03:53:16.357196 27210 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.26.146.148:37089
Aug 08, 2026 3:53:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 3:53:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 3:53:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 3:53:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 3:53:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 3:53:17 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 03:53:18 dist-test-slave-jlzn krb5kdc[10479](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786161198, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.146.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:53:18.828198 27210 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.896s user 0.142s sys 0.384s
I20260808 03:53:18.828686 27210 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 03:53:18.828928 27210 mini_ranger_kms.cc:401] 127.26.146.148:37089/kms/v1/keys
Aug 08 03:53:19 dist-test-slave-jlzn krb5kdc[10479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161198, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.146.148@KRBTEST.COM for HTTP/127.26.146.148@KRBTEST.COM
Aug 08 03:53:19 dist-test-slave-jlzn krb5kdc[10479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161198, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.146.148@KRBTEST.COM for HTTP/127.26.146.148@KRBTEST.COM
Aug 08 03:53:21 dist-test-slave-jlzn krb5kdc[10479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:53:21.733294 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:36803
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:38193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:36803
--ranger_config_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.146.148:37089/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:53:22.166175 11136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:53:22.166497 11136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:53:22.166831 11136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:53:22.176017 11136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:53:22.176134 11136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:53:22.176208 11136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:53:22.176375 11136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:53:22.176486 11136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:53:22.188563 11136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:38193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/master-0/wal
--ranger_kms_url=127.26.146.148:37089/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:36803
--ranger_config_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:36803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:53:22.189925 11136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:53:22.191370 11136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:53:22.205523 11146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:53:22.205464 11143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:53:22.205493 11144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:53:22.206208 11136 server_base.cc:1034] running on GCE node
I20260808 03:53:22.207074 11136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:53:22.208509 11136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:53:22.209780 11136 hybrid_clock.cc:648] HybridClock initialized: now 1786161202209676 us; error 35 us; skew 500 ppm
I20260808 03:53:22.210206 11136 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:53:22 dist-test-slave-jlzn krb5kdc[10479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161202, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:53:22.213840 11136 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:53:22.215438 11136 webserver.cc:460] Webserver started at http://127.26.146.190:42481/ using document root <none> and password file <none>
I20260808 03:53:22.216169 11136 fs_manager.cc:359] Metadata directory not provided
I20260808 03:53:22.216285 11136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:53:22.216571 11136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 03:53:22 dist-test-slave-jlzn krb5kdc[10479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161202, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for HTTP/127.26.146.148@KRBTEST.COM
I20260808 03:53:22.277812 11136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "59805368628d4fd99017069684291f53"
format_stamp: "Formatted at 2026-08-08 03:53:22 on dist-test-slave-jlzn"
server_key: "b33a79bd6aeb61d97b49d525458e59bf"
server_key_iv: "88ec4ee6fb255777017d42c5393e5d4d"
server_key_version: "kuduclusterkey@0"
I20260808 03:53:22.278956 11136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "59805368628d4fd99017069684291f53"
format_stamp: "Formatted at 2026-08-08 03:53:22 on dist-test-slave-jlzn"
server_key: "b33a79bd6aeb61d97b49d525458e59bf"
server_key_iv: "88ec4ee6fb255777017d42c5393e5d4d"
server_key_version: "kuduclusterkey@0"
I20260808 03:53:22.287000 11136 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.009s
I20260808 03:53:22.308518 11156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:53:22.310458 11136 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260808 03:53:22.310734 11136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "59805368628d4fd99017069684291f53"
format_stamp: "Formatted at 2026-08-08 03:53:22 on dist-test-slave-jlzn"
server_key: "b33a79bd6aeb61d97b49d525458e59bf"
server_key_iv: "88ec4ee6fb255777017d42c5393e5d4d"
server_key_version: "kuduclusterkey@0"
I20260808 03:53:22.311136 11136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:53:22.344532 11136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:53:22.350992 11136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:53:22.351364 11136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:53:22.381294 11136 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:36803
I20260808 03:53:22.381242 11232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:36803 every 8 connection(s)
I20260808 03:53:22.383298 11136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:53:22.385119 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 11136
I20260808 03:53:22.385331 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:53:22.386142 27210 external_mini_cluster.cc:1428] Setting key 9910539740c14bf35163ff0f6fa47395
I20260808 03:53:22.392982 11233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 03:53:22 dist-test-slave-jlzn krb5kdc[10479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:53:22.407071 11233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59805368628d4fd99017069684291f53: Bootstrap starting.
I20260808 03:53:22.412279 11233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59805368628d4fd99017069684291f53: Neither blocks nor log segments found. Creating new log.
I20260808 03:53:22.413793 11233 log.cc:824] T 00000000000000000000000000000000 P 59805368628d4fd99017069684291f53: Log is configured to *not* fsync() on all Append() calls
I20260808 03:53:22.415431 11236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:53:22.388390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35750 (local address 127.26.146.190:36803)
0808 03:53:22.390849 (+ 2459us) server_negotiation.cc:206] Beginning negotiation
0808 03:53:22.390866 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 03:53:22.392442 (+ 1576us) server_negotiation.cc:407] Connection header received
0808 03:53:22.394809 (+ 2367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:53:22.394852 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:53:22.395224 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:53:22.395630 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:53:22.397144 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:22.400014 (+ 2870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:22.401305 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:22.402417 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:22.402610 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:53:22.406464 (+ 3854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:53:22.406477 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:53:22.406497 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:53:22.406542 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:53:22.410855 (+ 4313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:53:22.411576 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:53:22.411584 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:53:22.411592 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:53:22.411787 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:53:22.412186 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:53:22.412191 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:53:22.412194 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:53:22.412944 (+ 750us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:53:22.413312 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0808 03:53:22.413623 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 03:53:22.414024 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2216,"thread_start_us":1941,"threads_started":1}
I20260808 03:53:22.417399 11233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59805368628d4fd99017069684291f53: No bootstrap required, opened a new log
I20260808 03:53:22.426446 11233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59805368628d4fd99017069684291f53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59805368628d4fd99017069684291f53" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36803 } }
I20260808 03:53:22.426941 11233 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59805368628d4fd99017069684291f53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:53:22.427057 11233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59805368628d4fd99017069684291f53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59805368628d4fd99017069684291f53, State: Initialized, Role: FOLLOWER
I20260808 03:53:22.427575 11233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59805368628d4fd99017069684291f53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59805368628d4fd99017069684291f53" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36803 } }
I20260808 03:53:22.427842 11233 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59805368628d4fd99017069684291f53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:53:22.428030 11233 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59805368628d4fd99017069684291f53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:53:22.428228 11233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59805368628d4fd99017069684291f53 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:53:22.430962 11233 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59805368628d4fd99017069684291f53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59805368628d4fd99017069684291f53" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36803 } }
I20260808 03:53:22.431633 11233 leader_election.cc:302] T 00000000000000000000000000000000 P 59805368628d4fd99017069684291f53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59805368628d4fd99017069684291f53; no voters:
I20260808 03:53:22.432188 11233 leader_election.cc:288] T 00000000000000000000000000000000 P 59805368628d4fd99017069684291f53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:53:22.432380 11242 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59805368628d4fd99017069684291f53 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:53:22.433506 11242 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59805368628d4fd99017069684291f53 [term 1 LEADER]: Becoming Leader. State: Replica: 59805368628d4fd99017069684291f53, State: Running, Role: LEADER
I20260808 03:53:22.434279 11242 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59805368628d4fd99017069684291f53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59805368628d4fd99017069684291f53" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36803 } }
I20260808 03:53:22.434510 11233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59805368628d4fd99017069684291f53 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:53:22.437613 11245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59805368628d4fd99017069684291f53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59805368628d4fd99017069684291f53. Latest consensus state: current_term: 1 leader_uuid: "59805368628d4fd99017069684291f53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59805368628d4fd99017069684291f53" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36803 } } }
I20260808 03:53:22.437846 11244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59805368628d4fd99017069684291f53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59805368628d4fd99017069684291f53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59805368628d4fd99017069684291f53" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36803 } } }
I20260808 03:53:22.438447 11245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59805368628d4fd99017069684291f53 [sys.catalog]: This master's current role is: LEADER
I20260808 03:53:22.438620 11244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59805368628d4fd99017069684291f53 [sys.catalog]: This master's current role is: LEADER
I20260808 03:53:22.443966 11233 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 03:53:22.444525 11250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:53:22.459731 11250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:53:22.473614 11250 catalog_manager.cc:1347] Generated new cluster ID: d4e58d1e1d2949d6b253c5b0912e7cbf
I20260808 03:53:22.473778 11250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:53:22.491393 11250 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:53:22.492969 11250 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:53:22.505460 11250 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59805368628d4fd99017069684291f53: Generated new TSK 0
I20260808 03:53:22.506366 11250 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
I20260808 03:53:23.264235 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.146.148:37089/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:36803
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:38193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 03:53:23 dist-test-slave-jlzn krb5kdc[10479](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786161203, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 03:53:23.665428 11308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:53:23.665730 11308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:53:23.665905 11308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:53:23.673316 11308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:53:23.673431 11308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:53:23.673594 11308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:53:23.683705 11308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:38193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.26.146.148:37089/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:36803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:53:23.685071 11308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:53:23.686614 11308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:53:23.704198 11324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:53:23.705484 11323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:53:23.724138 11326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:53:23.725617 11308 server_base.cc:1034] running on GCE node
I20260808 03:53:23.726491 11308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:53:23.727866 11308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:53:23.729141 11308 hybrid_clock.cc:648] HybridClock initialized: now 1786161203729073 us; error 50 us; skew 500 ppm
I20260808 03:53:23.729544 11308 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:53:23 dist-test-slave-jlzn krb5kdc[10479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161203, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:53:23.733958 11308 init.cc:377] Logged in from keytab as kudu/127.26.146.129@KRBTEST.COM (short username kudu)
I20260808 03:53:23.736251 11308 webserver.cc:460] Webserver started at http://127.26.146.129:42299/ using document root <none> and password file <none>
I20260808 03:53:23.737493 11308 fs_manager.cc:359] Metadata directory not provided
I20260808 03:53:23.737653 11308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:53:23.738066 11308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 03:53:23 dist-test-slave-jlzn krb5kdc[10479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161203, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for HTTP/127.26.146.148@KRBTEST.COM
I20260808 03:53:23.757740 11308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "6e3d3dbfb02b458dbf81a3966beabb29"
format_stamp: "Formatted at 2026-08-08 03:53:23 on dist-test-slave-jlzn"
server_key: "7d1d9d3fa88181bbe9dc5affeaec6b6f"
server_key_iv: "7bdc8fee606b2a01d29f627535336369"
server_key_version: "kuduclusterkey@0"
I20260808 03:53:23.759873 11308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "6e3d3dbfb02b458dbf81a3966beabb29"
format_stamp: "Formatted at 2026-08-08 03:53:23 on dist-test-slave-jlzn"
server_key: "7d1d9d3fa88181bbe9dc5affeaec6b6f"
server_key_iv: "7bdc8fee606b2a01d29f627535336369"
server_key_version: "kuduclusterkey@0"
I20260808 03:53:23.775434 11308 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.007s sys 0.007s
I20260808 03:53:23.806211 11339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:53:23.808280 11308 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.009s sys 0.000s
I20260808 03:53:23.808571 11308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "6e3d3dbfb02b458dbf81a3966beabb29"
format_stamp: "Formatted at 2026-08-08 03:53:23 on dist-test-slave-jlzn"
server_key: "7d1d9d3fa88181bbe9dc5affeaec6b6f"
server_key_iv: "7bdc8fee606b2a01d29f627535336369"
server_key_version: "kuduclusterkey@0"
I20260808 03:53:23.808918 11308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:53:23.834854 11308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:53:23.840628 11308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:53:23.840998 11308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:53:23.842309 11308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:53:23.844174 11308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:53:23.844327 11308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:53:23.844518 11308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:53:23.844648 11308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 03:53:24 dist-test-slave-jlzn krb5kdc[10479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161203, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for HTTP/127.26.146.148@KRBTEST.COM
I20260808 03:53:24.244421 11308 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:32865
I20260808 03:53:24.244673 11501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:32865 every 8 connection(s)
I20260808 03:53:24.246016 11308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:53:24.254885 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 11308
I20260808 03:53:24.255105 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:53:24.255931 27210 external_mini_cluster.cc:1428] Setting key 5737b71582abab91c3f670d5c0c64145
Aug 08 03:53:24 dist-test-slave-jlzn krb5kdc[10479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161203, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:53:24.284785 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:53:24.250808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36803 (local address 127.26.146.129:45657)
0808 03:53:24.252553 (+ 1745us) negotiation.cc:107] Waiting for socket to connect
0808 03:53:24.252607 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 03:53:24.255882 (+ 3275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:53:24.257082 (+ 1200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:53:24.257097 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:53:24.258005 (+ 908us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:53:24.259194 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:53:24.259219 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:53:24.261020 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:53:24.261025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:53:24.263701 (+ 2676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:53:24.263718 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:53:24.265427 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:53:24.265436 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:53:24.265835 (+ 399us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:53:24.270004 (+ 4169us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:53:24.270037 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:53:24.273680 (+ 3643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:53:24.278940 (+ 5260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:53:24.278949 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:53:24.278969 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:53:24.279677 (+ 708us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:53:24.280253 (+ 576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:53:24.280257 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:53:24.280261 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:53:24.280591 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:53:24.281481 (+ 890us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:53:24.281489 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:53:24.282001 (+ 512us) client_negotiation.cc:769] Sending connection context
0808 03:53:24.282357 (+ 356us) client_negotiation.cc:240] Negotiation successful
0808 03:53:24.283009 (+ 652us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1120,"thread_start_us":725,"threads_started":1}
I20260808 03:53:24.286172 11505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:53:24.249852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:45657 (local address 127.26.146.190:36803)
0808 03:53:24.250804 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0808 03:53:24.250813 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:53:24.252726 (+ 1913us) server_negotiation.cc:407] Connection header received
0808 03:53:24.256214 (+ 3488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:53:24.256220 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:53:24.256306 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:53:24.256428 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:53:24.259382 (+ 2954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:24.260802 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:24.263920 (+ 3118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:24.264854 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:24.264951 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:53:24.274015 (+ 9064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:53:24.274024 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:53:24.274029 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:53:24.274071 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:53:24.278759 (+ 4688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:53:24.279849 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:53:24.279854 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:53:24.279857 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:53:24.280105 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:53:24.280764 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:53:24.280768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:53:24.280771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:53:24.281279 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:53:24.281401 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 03:53:24.285401 (+ 4000us) server_negotiation.cc:299] Negotiation successful
0808 03:53:24.285611 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":779,"thread_start_us":498,"threads_started":1}
I20260808 03:53:24.288748 11502 heartbeater.cc:347] Connected to a master server at 127.26.146.190:36803
I20260808 03:53:24.289227 11502 heartbeater.cc:464] Registering TS with master...
I20260808 03:53:24.290548 11502 heartbeater.cc:511] Master 127.26.146.190:36803 requested a full tablet report, sending...
I20260808 03:53:24.293792 11185 ts_manager.cc:194] Registered new tserver with Master: 6e3d3dbfb02b458dbf81a3966beabb29 (127.26.146.129:32865)
WARNING: no policy specified for kudu/127.26.146.130@KRBTEST.COM; defaulting to no policy
I20260808 03:53:24.296517 11185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.129@KRBTEST.COM'} at 127.26.146.129:45657
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
I20260808 03:53:24.331396 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.146.148:37089/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:36803
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:38193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 03:53:24 dist-test-slave-jlzn krb5kdc[10479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161203, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for HTTP/127.26.146.148@KRBTEST.COM
W20260808 03:53:25.076508 11511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:53:25.076884 11511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:53:25.077107 11511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:53:25.087461 11511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:53:25.087821 11511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:53:25.088078 11511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:53:25.100975 11511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:38193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.26.146.148:37089/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:36803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:53:25.102746 11511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:53:25.104302 11511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:53:25.125633 11533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:53:25.126199 11534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:53:25.126492 11536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:53:25.131767 11511 server_base.cc:1034] running on GCE node
I20260808 03:53:25.132699 11511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:53:25.134274 11511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:53:25.135641 11511 hybrid_clock.cc:648] HybridClock initialized: now 1786161205135507 us; error 41 us; skew 500 ppm
I20260808 03:53:25.136240 11511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:53:25 dist-test-slave-jlzn krb5kdc[10479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161205, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:53:25.142403 11511 init.cc:377] Logged in from keytab as kudu/127.26.146.130@KRBTEST.COM (short username kudu)
I20260808 03:53:25.145187 11511 webserver.cc:460] Webserver started at http://127.26.146.130:43255/ using document root <none> and password file <none>
I20260808 03:53:25.146337 11511 fs_manager.cc:359] Metadata directory not provided
I20260808 03:53:25.146576 11511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:53:25.147091 11511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 03:53:25 dist-test-slave-jlzn krb5kdc[10479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161205, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for HTTP/127.26.146.148@KRBTEST.COM
I20260808 03:53:25.168602 11511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "9cf25e812601404e92d7250c60399b0f"
format_stamp: "Formatted at 2026-08-08 03:53:25 on dist-test-slave-jlzn"
server_key: "d0376686cfc1cb61538f0f02e760fdca"
server_key_iv: "8123510d66ea3209e6f3c35464d23123"
server_key_version: "kuduclusterkey@0"
I20260808 03:53:25.169759 11511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "9cf25e812601404e92d7250c60399b0f"
format_stamp: "Formatted at 2026-08-08 03:53:25 on dist-test-slave-jlzn"
server_key: "d0376686cfc1cb61538f0f02e760fdca"
server_key_iv: "8123510d66ea3209e6f3c35464d23123"
server_key_version: "kuduclusterkey@0"
I20260808 03:53:25.178347 11511 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.004s
I20260808 03:53:25.194695 11547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:53:25.196516 11511 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260808 03:53:25.196758 11511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "9cf25e812601404e92d7250c60399b0f"
format_stamp: "Formatted at 2026-08-08 03:53:25 on dist-test-slave-jlzn"
server_key: "d0376686cfc1cb61538f0f02e760fdca"
server_key_iv: "8123510d66ea3209e6f3c35464d23123"
server_key_version: "kuduclusterkey@0"
I20260808 03:53:25.197054 11511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:53:25.215413 11511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:53:25.221335 11511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:53:25.221813 11511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:53:25.223121 11511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:53:25.225168 11511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:53:25.225318 11511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:53:25.225513 11511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:53:25.225646 11511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:53:25.301132 11502 heartbeater.cc:503] Master 127.26.146.190:36803 was elected leader, sending a full tablet report...
I20260808 03:53:25.320499 11511 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:42443
I20260808 03:53:25.320567 11719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:42443 every 8 connection(s)
I20260808 03:53:25.322192 11511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:53:25.330807 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 11511
I20260808 03:53:25.330978 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:53:25.331542 27210 external_mini_cluster.cc:1428] Setting key fa1d4cace5ebe14b79a52528cd4ad7e0
Aug 08 03:53:25 dist-test-slave-jlzn krb5kdc[10479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161205, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:53:25.356196 11723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:53:25.326251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:51863 (local address 127.26.146.190:36803)
0808 03:53:25.327148 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0808 03:53:25.327155 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:53:25.328660 (+ 1505us) server_negotiation.cc:407] Connection header received
0808 03:53:25.331986 (+ 3326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:53:25.331993 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:53:25.332149 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:53:25.332275 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:53:25.334976 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:25.336454 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:25.339453 (+ 2999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:25.340476 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:25.340644 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:53:25.346493 (+ 5849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:53:25.346501 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:53:25.346506 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:53:25.346548 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:53:25.351368 (+ 4820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:53:25.352547 (+ 1179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:53:25.352553 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:53:25.352557 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:53:25.352792 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:53:25.353468 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:53:25.353474 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:53:25.353477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:53:25.353944 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:53:25.354085 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 03:53:25.355310 (+ 1225us) server_negotiation.cc:299] Negotiation successful
0808 03:53:25.355589 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":473,"threads_started":1}
I20260808 03:53:25.358373 11725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:53:25.327047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36803 (local address 127.26.146.130:51863)
0808 03:53:25.328471 (+ 1424us) negotiation.cc:107] Waiting for socket to connect
0808 03:53:25.328533 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 03:53:25.331649 (+ 3116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:53:25.332555 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:53:25.332566 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:53:25.333440 (+ 874us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:53:25.334681 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:53:25.334709 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:53:25.336636 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:53:25.336642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:53:25.339266 (+ 2624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:53:25.339282 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:53:25.340913 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:53:25.340925 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:53:25.341357 (+ 432us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:53:25.342299 (+ 942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:53:25.342334 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:53:25.346212 (+ 3878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:53:25.351560 (+ 5348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:53:25.351570 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:53:25.351588 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:53:25.352404 (+ 816us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:53:25.352949 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:53:25.352954 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:53:25.352958 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:53:25.353329 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:53:25.354199 (+ 870us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:53:25.354207 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:53:25.354741 (+ 534us) client_negotiation.cc:769] Sending connection context
0808 03:53:25.355913 (+ 1172us) client_negotiation.cc:240] Negotiation successful
0808 03:53:25.356604 (+ 691us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1061,"thread_start_us":729,"threads_started":1}
I20260808 03:53:25.360451 11720 heartbeater.cc:347] Connected to a master server at 127.26.146.190:36803
I20260808 03:53:25.360895 11720 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.26.146.131@KRBTEST.COM; defaulting to no policy
I20260808 03:53:25.362308 11720 heartbeater.cc:511] Master 127.26.146.190:36803 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.131@KRBTEST.COM" created.
I20260808 03:53:25.364586 11185 ts_manager.cc:194] Registered new tserver with Master: 9cf25e812601404e92d7250c60399b0f (127.26.146.130:42443)
I20260808 03:53:25.365525 11185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.130@KRBTEST.COM'} at 127.26.146.130:51863
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
I20260808 03:53:25.398772 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.146.148:37089/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:36803
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:38193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:53:26.042634 11729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:53:26.043035 11729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:53:26.043273 11729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:53:26.054291 11729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:53:26.054492 11729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:53:26.054742 11729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:53:26.067996 11729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:38193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.26.146.148:37089/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:36803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:53:26.069599 11729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:53:26.071148 11729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:53:26.089664 11735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:53:26.090070 11739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:53:26.090143 11736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:53:26.091305 11729 server_base.cc:1034] running on GCE node
I20260808 03:53:26.092108 11729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:53:26.093279 11729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:53:26.094530 11729 hybrid_clock.cc:648] HybridClock initialized: now 1786161206094435 us; error 37 us; skew 500 ppm
I20260808 03:53:26.094929 11729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:53:26 dist-test-slave-jlzn krb5kdc[10479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161206, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:53:26.099645 11729 init.cc:377] Logged in from keytab as kudu/127.26.146.131@KRBTEST.COM (short username kudu)
I20260808 03:53:26.101724 11729 webserver.cc:460] Webserver started at http://127.26.146.131:42727/ using document root <none> and password file <none>
I20260808 03:53:26.102654 11729 fs_manager.cc:359] Metadata directory not provided
I20260808 03:53:26.102854 11729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:53:26.103271 11729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 03:53:26 dist-test-slave-jlzn krb5kdc[10479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161206, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for HTTP/127.26.146.148@KRBTEST.COM
I20260808 03:53:26.124109 11729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "6bd1ecdea73241039b18cfcdd2d19294"
format_stamp: "Formatted at 2026-08-08 03:53:26 on dist-test-slave-jlzn"
server_key: "d70be9b27b969971d63a430f9bf2255a"
server_key_iv: "42f56498d785be4504112ee0e9e6d914"
server_key_version: "kuduclusterkey@0"
I20260808 03:53:26.125548 11729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "6bd1ecdea73241039b18cfcdd2d19294"
format_stamp: "Formatted at 2026-08-08 03:53:26 on dist-test-slave-jlzn"
server_key: "d70be9b27b969971d63a430f9bf2255a"
server_key_iv: "42f56498d785be4504112ee0e9e6d914"
server_key_version: "kuduclusterkey@0"
I20260808 03:53:26.135035 11729 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.005s sys 0.005s
I20260808 03:53:26.155395 11751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:53:26.157083 11729 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260808 03:53:26.157344 11729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "6bd1ecdea73241039b18cfcdd2d19294"
format_stamp: "Formatted at 2026-08-08 03:53:26 on dist-test-slave-jlzn"
server_key: "d70be9b27b969971d63a430f9bf2255a"
server_key_iv: "42f56498d785be4504112ee0e9e6d914"
server_key_version: "kuduclusterkey@0"
I20260808 03:53:26.157647 11729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:53:26.189040 11729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:53:26.194527 11729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:53:26.195039 11729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:53:26.196698 11729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:53:26.198990 11729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:53:26.199113 11729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:53:26.199318 11729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:53:26.199436 11729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:53:26.277705 11729 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:39677
I20260808 03:53:26.277769 11935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:39677 every 8 connection(s)
I20260808 03:53:26.279302 11729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:53:26.281908 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 11729
I20260808 03:53:26.282086 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:53:26.282577 27210 external_mini_cluster.cc:1428] Setting key fd21c39851bcb35bfc106925b1d80f70
Aug 08 03:53:26 dist-test-slave-jlzn krb5kdc[10479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161206, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:53:26.306782 11940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:53:26.283175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:37497 (local address 127.26.146.190:36803)
0808 03:53:26.284278 (+ 1103us) server_negotiation.cc:206] Beginning negotiation
0808 03:53:26.284285 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:53:26.285687 (+ 1402us) server_negotiation.cc:407] Connection header received
0808 03:53:26.288797 (+ 3110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:53:26.288802 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:53:26.288884 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:53:26.288980 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:53:26.290991 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.291979 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.294076 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.294716 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.294807 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:53:26.299478 (+ 4671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:53:26.299485 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:53:26.299489 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:53:26.299528 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:53:26.303555 (+ 4027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:53:26.304470 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:53:26.304474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:53:26.304477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:53:26.304653 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:53:26.305108 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:53:26.305112 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:53:26.305115 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:53:26.305506 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:53:26.305613 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 03:53:26.306258 (+ 645us) server_negotiation.cc:299] Negotiation successful
0808 03:53:26.306449 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":937,"thread_start_us":723,"threads_started":1}
I20260808 03:53:26.307912 11943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:53:26.284290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36803 (local address 127.26.146.131:37497)
0808 03:53:26.285523 (+ 1233us) negotiation.cc:107] Waiting for socket to connect
0808 03:53:26.285563 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 03:53:26.288548 (+ 2985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:53:26.289254 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:53:26.289262 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:53:26.289974 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:53:26.290819 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:53:26.290839 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.292124 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.292128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:53:26.293933 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:53:26.293946 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.294961 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.294967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:53:26.295274 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:53:26.296147 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:53:26.296174 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:53:26.299242 (+ 3068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:53:26.303667 (+ 4425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:53:26.303674 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:53:26.303685 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:53:26.304350 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:53:26.304780 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:53:26.304783 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:53:26.304786 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:53:26.304985 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:53:26.305649 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:53:26.305654 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:53:26.305958 (+ 304us) client_negotiation.cc:769] Sending connection context
0808 03:53:26.306184 (+ 226us) client_negotiation.cc:240] Negotiation successful
0808 03:53:26.306595 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":959,"thread_start_us":679,"threads_started":1}
I20260808 03:53:26.309326 11936 heartbeater.cc:347] Connected to a master server at 127.26.146.190:36803
I20260808 03:53:26.309633 11936 heartbeater.cc:464] Registering TS with master...
I20260808 03:53:26.310557 11936 heartbeater.cc:511] Master 127.26.146.190:36803 requested a full tablet report, sending...
I20260808 03:53:26.312412 11183 ts_manager.cc:194] Registered new tserver with Master: 6bd1ecdea73241039b18cfcdd2d19294 (127.26.146.131:39677)
I20260808 03:53:26.313692 11183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.131@KRBTEST.COM'} at 127.26.146.131:37497
I20260808 03:53:26.324762 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:53:26.347297 11940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:53:26.330831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35762 (local address 127.26.146.190:36803)
0808 03:53:26.331043 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 03:53:26.331050 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:53:26.331478 (+ 428us) server_negotiation.cc:407] Connection header received
0808 03:53:26.331668 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:53:26.331671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:53:26.331781 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:53:26.331899 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:53:26.334165 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.335186 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.337197 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.337835 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.337927 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:53:26.340980 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:53:26.340987 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:53:26.340990 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:53:26.341021 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:53:26.344401 (+ 3380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:53:26.345140 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:53:26.345143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:53:26.345145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:53:26.345329 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:53:26.345726 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:53:26.345729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:53:26.345731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:53:26.346054 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:53:26.346294 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 03:53:26.346662 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 03:53:26.346939 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 03:53:26.351945 11183 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35762:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:53:26.362404 11183 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:53:26.368124 11720 heartbeater.cc:503] Master 127.26.146.190:36803 was elected leader, sending a full tablet report...
I20260808 03:53:26.421343 11963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:53:26.379869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34060 (local address 127.26.146.130:42443)
0808 03:53:26.405111 (+ 25242us) server_negotiation.cc:206] Beginning negotiation
0808 03:53:26.405123 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 03:53:26.409585 (+ 4462us) server_negotiation.cc:407] Connection header received
0808 03:53:26.409712 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:53:26.409720 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:53:26.409930 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:53:26.410140 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:53:26.412053 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.414479 (+ 2426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.418786 (+ 4307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.420046 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.420232 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:53:26.420424 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0808 03:53:26.420839 (+ 415us) server_negotiation.cc:299] Negotiation successful
0808 03:53:26.420991 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25031,"thread_start_us":24762,"threads_started":1}
I20260808 03:53:26.421619 11966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:53:26.379928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:42443 (local address 127.0.0.1:34060)
0808 03:53:26.408548 (+ 28620us) negotiation.cc:107] Waiting for socket to connect
0808 03:53:26.408606 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 03:53:26.408852 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:53:26.410615 (+ 1763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:53:26.410623 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:53:26.410655 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:53:26.411734 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:53:26.411749 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.414706 (+ 2957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.414711 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:53:26.418620 (+ 3909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:53:26.418661 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.420328 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.420336 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:53:26.420583 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:53:26.420645 (+ 62us) client_negotiation.cc:769] Sending connection context
0808 03:53:26.420750 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 03:53:26.420935 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28183,"mutex_wait_us":2,"thread_start_us":23733,"threads_started":1}
I20260808 03:53:26.423588 11967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:53:26.380197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:39677 (local address 127.0.0.1:57062)
0808 03:53:26.409270 (+ 29073us) negotiation.cc:107] Waiting for socket to connect
0808 03:53:26.409289 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 03:53:26.409447 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:53:26.410861 (+ 1414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:53:26.410870 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:53:26.410904 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:53:26.414022 (+ 3118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:53:26.414039 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.416284 (+ 2245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.416290 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:53:26.419336 (+ 3046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:53:26.419351 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.422353 (+ 3002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.422361 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:53:26.422626 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:53:26.422659 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 03:53:26.422743 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 03:53:26.422907 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28924,"mutex_wait_us":25,"thread_start_us":27070,"threads_started":1}
I20260808 03:53:26.424635 11962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:53:26.379663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57062 (local address 127.26.146.131:39677)
0808 03:53:26.395169 (+ 15506us) server_negotiation.cc:206] Beginning negotiation
0808 03:53:26.395179 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:53:26.409425 (+ 14246us) server_negotiation.cc:407] Connection header received
0808 03:53:26.409608 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:53:26.409618 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:53:26.409952 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:53:26.410202 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:53:26.414289 (+ 4087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.416148 (+ 1859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.419512 (+ 3364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.421656 (+ 2144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.421907 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:53:26.422112 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 03:53:26.423860 (+ 1748us) server_negotiation.cc:299] Negotiation successful
0808 03:53:26.424050 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15234,"thread_start_us":15510,"threads_started":1}
I20260808 03:53:26.426110 11969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:53:26.384871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57574 (local address 127.26.146.129:32865)
0808 03:53:26.405935 (+ 21064us) server_negotiation.cc:206] Beginning negotiation
0808 03:53:26.405943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:53:26.412989 (+ 7046us) server_negotiation.cc:407] Connection header received
0808 03:53:26.414463 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:53:26.414471 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:53:26.414760 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:53:26.415005 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:53:26.416446 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.418266 (+ 1820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.421681 (+ 3415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.424818 (+ 3137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.425078 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:53:26.425213 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 03:53:26.425579 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 03:53:26.425717 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20822,"thread_start_us":20567,"threads_started":1}
I20260808 03:53:26.426179 11968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:53:26.383917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:32865 (local address 127.0.0.1:57574)
0808 03:53:26.412816 (+ 28899us) negotiation.cc:107] Waiting for socket to connect
0808 03:53:26.412836 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:53:26.414254 (+ 1418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:53:26.415003 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:53:26.415009 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:53:26.415049 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:53:26.416150 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:53:26.416171 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.418451 (+ 2280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.418456 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:53:26.421450 (+ 2994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:53:26.421470 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.425156 (+ 3686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.425165 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:53:26.425406 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:53:26.425439 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 03:53:26.425530 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 03:53:26.425649 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28681,"thread_start_us":28215,"threads_started":1}
I20260808 03:53:26.426666 11618 tablet_service.cc:1467] Processing CreateTablet for tablet 2d78be7fe57f423d8d02864f68a0ff08 (DEFAULT_TABLE table=test-table [id=4c55c7fed42948568328aebcaf4c5e4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:53:26.428449 11618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d78be7fe57f423d8d02864f68a0ff08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:53:26.430080 11412 tablet_service.cc:1467] Processing CreateTablet for tablet 2d78be7fe57f423d8d02864f68a0ff08 (DEFAULT_TABLE table=test-table [id=4c55c7fed42948568328aebcaf4c5e4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:53:26.431797 11412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d78be7fe57f423d8d02864f68a0ff08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:53:26.439289 11826 tablet_service.cc:1467] Processing CreateTablet for tablet 2d78be7fe57f423d8d02864f68a0ff08 (DEFAULT_TABLE table=test-table [id=4c55c7fed42948568328aebcaf4c5e4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:53:26.441365 11826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d78be7fe57f423d8d02864f68a0ff08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:53:26.445721 11971 tablet_bootstrap.cc:492] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f: Bootstrap starting.
I20260808 03:53:26.450711 11971 tablet_bootstrap.cc:654] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f: Neither blocks nor log segments found. Creating new log.
I20260808 03:53:26.451113 11972 tablet_bootstrap.cc:492] T 2d78be7fe57f423d8d02864f68a0ff08 P 6e3d3dbfb02b458dbf81a3966beabb29: Bootstrap starting.
I20260808 03:53:26.453069 11971 log.cc:824] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f: Log is configured to *not* fsync() on all Append() calls
I20260808 03:53:26.457348 11972 tablet_bootstrap.cc:654] T 2d78be7fe57f423d8d02864f68a0ff08 P 6e3d3dbfb02b458dbf81a3966beabb29: Neither blocks nor log segments found. Creating new log.
I20260808 03:53:26.457963 11971 tablet_bootstrap.cc:492] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f: No bootstrap required, opened a new log
I20260808 03:53:26.458370 11971 ts_tablet_manager.cc:1397] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.009s
I20260808 03:53:26.459592 11972 log.cc:824] T 2d78be7fe57f423d8d02864f68a0ff08 P 6e3d3dbfb02b458dbf81a3966beabb29: Log is configured to *not* fsync() on all Append() calls
I20260808 03:53:26.460510 11974 tablet_bootstrap.cc:492] T 2d78be7fe57f423d8d02864f68a0ff08 P 6bd1ecdea73241039b18cfcdd2d19294: Bootstrap starting.
I20260808 03:53:26.464704 11972 tablet_bootstrap.cc:492] T 2d78be7fe57f423d8d02864f68a0ff08 P 6e3d3dbfb02b458dbf81a3966beabb29: No bootstrap required, opened a new log
I20260808 03:53:26.465107 11972 ts_tablet_manager.cc:1397] T 2d78be7fe57f423d8d02864f68a0ff08 P 6e3d3dbfb02b458dbf81a3966beabb29: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.005s
I20260808 03:53:26.465476 11974 tablet_bootstrap.cc:654] T 2d78be7fe57f423d8d02864f68a0ff08 P 6bd1ecdea73241039b18cfcdd2d19294: Neither blocks nor log segments found. Creating new log.
I20260808 03:53:26.467211 11974 log.cc:824] T 2d78be7fe57f423d8d02864f68a0ff08 P 6bd1ecdea73241039b18cfcdd2d19294: Log is configured to *not* fsync() on all Append() calls
I20260808 03:53:26.471251 11974 tablet_bootstrap.cc:492] T 2d78be7fe57f423d8d02864f68a0ff08 P 6bd1ecdea73241039b18cfcdd2d19294: No bootstrap required, opened a new log
I20260808 03:53:26.471601 11974 ts_tablet_manager.cc:1397] T 2d78be7fe57f423d8d02864f68a0ff08 P 6bd1ecdea73241039b18cfcdd2d19294: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.002s
I20260808 03:53:26.473773 11971 raft_consensus.cc:348] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bd1ecdea73241039b18cfcdd2d19294" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 39677 } } peers { permanent_uuid: "9cf25e812601404e92d7250c60399b0f" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 42443 } } peers { permanent_uuid: "6e3d3dbfb02b458dbf81a3966beabb29" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32865 } }
I20260808 03:53:26.474423 11971 raft_consensus.cc:374] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:53:26.474608 11971 raft_consensus.cc:729] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cf25e812601404e92d7250c60399b0f, State: Initialized, Role: FOLLOWER
I20260808 03:53:26.475415 11971 consensus_queue.cc:260] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bd1ecdea73241039b18cfcdd2d19294" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 39677 } } peers { permanent_uuid: "9cf25e812601404e92d7250c60399b0f" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 42443 } } peers { permanent_uuid: "6e3d3dbfb02b458dbf81a3966beabb29" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32865 } }
I20260808 03:53:26.478071 11971 ts_tablet_manager.cc:1428] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f: Time spent starting tablet: real 0.019s user 0.008s sys 0.013s
I20260808 03:53:26.480888 11972 raft_consensus.cc:348] T 2d78be7fe57f423d8d02864f68a0ff08 P 6e3d3dbfb02b458dbf81a3966beabb29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bd1ecdea73241039b18cfcdd2d19294" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 39677 } } peers { permanent_uuid: "9cf25e812601404e92d7250c60399b0f" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 42443 } } peers { permanent_uuid: "6e3d3dbfb02b458dbf81a3966beabb29" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32865 } }
I20260808 03:53:26.481611 11972 raft_consensus.cc:374] T 2d78be7fe57f423d8d02864f68a0ff08 P 6e3d3dbfb02b458dbf81a3966beabb29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:53:26.481823 11972 raft_consensus.cc:729] T 2d78be7fe57f423d8d02864f68a0ff08 P 6e3d3dbfb02b458dbf81a3966beabb29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e3d3dbfb02b458dbf81a3966beabb29, State: Initialized, Role: FOLLOWER
I20260808 03:53:26.482625 11972 consensus_queue.cc:260] T 2d78be7fe57f423d8d02864f68a0ff08 P 6e3d3dbfb02b458dbf81a3966beabb29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bd1ecdea73241039b18cfcdd2d19294" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 39677 } } peers { permanent_uuid: "9cf25e812601404e92d7250c60399b0f" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 42443 } } peers { permanent_uuid: "6e3d3dbfb02b458dbf81a3966beabb29" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32865 } }
I20260808 03:53:26.484769 11972 ts_tablet_manager.cc:1428] T 2d78be7fe57f423d8d02864f68a0ff08 P 6e3d3dbfb02b458dbf81a3966beabb29: Time spent starting tablet: real 0.019s user 0.013s sys 0.007s
I20260808 03:53:26.485319 11974 raft_consensus.cc:348] T 2d78be7fe57f423d8d02864f68a0ff08 P 6bd1ecdea73241039b18cfcdd2d19294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bd1ecdea73241039b18cfcdd2d19294" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 39677 } } peers { permanent_uuid: "9cf25e812601404e92d7250c60399b0f" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 42443 } } peers { permanent_uuid: "6e3d3dbfb02b458dbf81a3966beabb29" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32865 } }
I20260808 03:53:26.485723 11974 raft_consensus.cc:374] T 2d78be7fe57f423d8d02864f68a0ff08 P 6bd1ecdea73241039b18cfcdd2d19294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:53:26.485896 11974 raft_consensus.cc:729] T 2d78be7fe57f423d8d02864f68a0ff08 P 6bd1ecdea73241039b18cfcdd2d19294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bd1ecdea73241039b18cfcdd2d19294, State: Initialized, Role: FOLLOWER
I20260808 03:53:26.486613 11974 consensus_queue.cc:260] T 2d78be7fe57f423d8d02864f68a0ff08 P 6bd1ecdea73241039b18cfcdd2d19294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bd1ecdea73241039b18cfcdd2d19294" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 39677 } } peers { permanent_uuid: "9cf25e812601404e92d7250c60399b0f" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 42443 } } peers { permanent_uuid: "6e3d3dbfb02b458dbf81a3966beabb29" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32865 } }
I20260808 03:53:26.488471 11936 heartbeater.cc:503] Master 127.26.146.190:36803 was elected leader, sending a full tablet report...
I20260808 03:53:26.488978 11974 ts_tablet_manager.cc:1428] T 2d78be7fe57f423d8d02864f68a0ff08 P 6bd1ecdea73241039b18cfcdd2d19294: Time spent starting tablet: real 0.017s user 0.005s sys 0.012s
W20260808 03:53:26.500195 11503 tablet.cc:2369] T 2d78be7fe57f423d8d02864f68a0ff08 P 6e3d3dbfb02b458dbf81a3966beabb29: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:53:26.532437 11939 tablet.cc:2369] T 2d78be7fe57f423d8d02864f68a0ff08 P 6bd1ecdea73241039b18cfcdd2d19294: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:53:26.560954 11978 raft_consensus.cc:482] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:53:26.561381 11978 raft_consensus.cc:504] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bd1ecdea73241039b18cfcdd2d19294" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 39677 } } peers { permanent_uuid: "9cf25e812601404e92d7250c60399b0f" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 42443 } } peers { permanent_uuid: "6e3d3dbfb02b458dbf81a3966beabb29" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32865 } }
I20260808 03:53:26.563866 11978 leader_election.cc:288] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bd1ecdea73241039b18cfcdd2d19294 (127.26.146.131:39677), 6e3d3dbfb02b458dbf81a3966beabb29 (127.26.146.129:32865)
W20260808 03:53:26.576009 11721 tablet.cc:2369] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:53:26.576504 11962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:53:26.564504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:56915 (local address 127.26.146.131:39677)
0808 03:53:26.564771 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0808 03:53:26.564778 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:53:26.566431 (+ 1653us) server_negotiation.cc:407] Connection header received
0808 03:53:26.566658 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:53:26.566664 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:53:26.566726 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:53:26.566800 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:53:26.568230 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.569653 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.573476 (+ 3823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.575279 (+ 1803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.575481 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:53:26.575563 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 03:53:26.575995 (+ 432us) server_negotiation.cc:299] Negotiation successful
0808 03:53:26.576166 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260808 03:53:26.576781 11969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:53:26.564443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:39709 (local address 127.26.146.129:32865)
0808 03:53:26.564709 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0808 03:53:26.564715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:53:26.566861 (+ 2146us) server_negotiation.cc:407] Connection header received
0808 03:53:26.567073 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:53:26.567077 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:53:26.567136 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:53:26.567203 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:53:26.568498 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.569903 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.573611 (+ 3708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.575393 (+ 1782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.576240 (+ 847us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:53:26.576321 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 03:53:26.576372 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 03:53:26.576486 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260808 03:53:26.576794 11985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:53:26.564930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:32865 (local address 127.26.146.130:39709)
0808 03:53:26.566771 (+ 1841us) negotiation.cc:107] Waiting for socket to connect
0808 03:53:26.566786 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 03:53:26.566917 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:53:26.567228 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:53:26.567231 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:53:26.567248 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:53:26.568279 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:53:26.568291 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.570063 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.570068 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:53:26.573433 (+ 3365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:53:26.573445 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.575683 (+ 2238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.575690 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:53:26.575884 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:53:26.575914 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 03:53:26.575984 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 03:53:26.576088 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1710,"mutex_wait_us":27,"thread_start_us":1448,"threads_started":1}
I20260808 03:53:26.577505 11982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:53:26.564559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:39677 (local address 127.26.146.130:56915)
0808 03:53:26.566305 (+ 1746us) negotiation.cc:107] Waiting for socket to connect
0808 03:53:26.566338 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 03:53:26.566511 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:53:26.566906 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:53:26.566912 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:53:26.566939 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:53:26.568090 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:53:26.568102 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.569821 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.569825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:53:26.573311 (+ 3486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:53:26.573327 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.575518 (+ 2191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.575525 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:53:26.575766 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:53:26.575831 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 03:53:26.575934 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 03:53:26.576089 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1600,"mutex_wait_us":27,"thread_start_us":1241,"threads_started":1}
I20260808 03:53:26.578365 11440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d78be7fe57f423d8d02864f68a0ff08" candidate_uuid: "9cf25e812601404e92d7250c60399b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e3d3dbfb02b458dbf81a3966beabb29" is_pre_election: true
I20260808 03:53:26.578692 11858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d78be7fe57f423d8d02864f68a0ff08" candidate_uuid: "9cf25e812601404e92d7250c60399b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bd1ecdea73241039b18cfcdd2d19294" is_pre_election: true
I20260808 03:53:26.579110 11440 raft_consensus.cc:2454] T 2d78be7fe57f423d8d02864f68a0ff08 P 6e3d3dbfb02b458dbf81a3966beabb29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cf25e812601404e92d7250c60399b0f in term 0.
I20260808 03:53:26.579416 11858 raft_consensus.cc:2454] T 2d78be7fe57f423d8d02864f68a0ff08 P 6bd1ecdea73241039b18cfcdd2d19294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cf25e812601404e92d7250c60399b0f in term 0.
I20260808 03:53:26.580155 11555 leader_election.cc:302] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e3d3dbfb02b458dbf81a3966beabb29, 9cf25e812601404e92d7250c60399b0f; no voters:
I20260808 03:53:26.580688 11978 raft_consensus.cc:2781] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:53:26.580919 11978 raft_consensus.cc:482] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:53:26.581070 11978 raft_consensus.cc:3037] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:53:26.584784 11978 raft_consensus.cc:504] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bd1ecdea73241039b18cfcdd2d19294" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 39677 } } peers { permanent_uuid: "9cf25e812601404e92d7250c60399b0f" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 42443 } } peers { permanent_uuid: "6e3d3dbfb02b458dbf81a3966beabb29" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32865 } }
I20260808 03:53:26.585675 11978 leader_election.cc:288] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f [CANDIDATE]: Term 1 election: Requested vote from peers 6bd1ecdea73241039b18cfcdd2d19294 (127.26.146.131:39677), 6e3d3dbfb02b458dbf81a3966beabb29 (127.26.146.129:32865)
I20260808 03:53:26.586262 11440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d78be7fe57f423d8d02864f68a0ff08" candidate_uuid: "9cf25e812601404e92d7250c60399b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e3d3dbfb02b458dbf81a3966beabb29"
I20260808 03:53:26.586246 11858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d78be7fe57f423d8d02864f68a0ff08" candidate_uuid: "9cf25e812601404e92d7250c60399b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bd1ecdea73241039b18cfcdd2d19294"
I20260808 03:53:26.586526 11858 raft_consensus.cc:3037] T 2d78be7fe57f423d8d02864f68a0ff08 P 6bd1ecdea73241039b18cfcdd2d19294 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:53:26.586531 11440 raft_consensus.cc:3037] T 2d78be7fe57f423d8d02864f68a0ff08 P 6e3d3dbfb02b458dbf81a3966beabb29 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:53:26.591169 11858 raft_consensus.cc:2454] T 2d78be7fe57f423d8d02864f68a0ff08 P 6bd1ecdea73241039b18cfcdd2d19294 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cf25e812601404e92d7250c60399b0f in term 1.
I20260808 03:53:26.591236 11440 raft_consensus.cc:2454] T 2d78be7fe57f423d8d02864f68a0ff08 P 6e3d3dbfb02b458dbf81a3966beabb29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cf25e812601404e92d7250c60399b0f in term 1.
I20260808 03:53:26.591750 11555 leader_election.cc:302] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6bd1ecdea73241039b18cfcdd2d19294, 6e3d3dbfb02b458dbf81a3966beabb29, 9cf25e812601404e92d7250c60399b0f; no voters:
I20260808 03:53:26.592166 11978 raft_consensus.cc:2781] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:53:26.592893 11978 raft_consensus.cc:686] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f [term 1 LEADER]: Becoming Leader. State: Replica: 9cf25e812601404e92d7250c60399b0f, State: Running, Role: LEADER
I20260808 03:53:26.593453 11978 consensus_queue.cc:237] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bd1ecdea73241039b18cfcdd2d19294" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 39677 } } peers { permanent_uuid: "9cf25e812601404e92d7250c60399b0f" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 42443 } } peers { permanent_uuid: "6e3d3dbfb02b458dbf81a3966beabb29" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32865 } }
I20260808 03:53:26.599047 11183 catalog_manager.cc:5495] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 9cf25e812601404e92d7250c60399b0f (127.26.146.130). New cstate: current_term: 1 leader_uuid: "9cf25e812601404e92d7250c60399b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bd1ecdea73241039b18cfcdd2d19294" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 39677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9cf25e812601404e92d7250c60399b0f" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 42443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e3d3dbfb02b458dbf81a3966beabb29" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32865 } health_report { overall_health: UNKNOWN } } }
I20260808 03:53:26.656104 11963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:53:26.649341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34066 (local address 127.26.146.130:42443)
0808 03:53:26.649583 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 03:53:26.649589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:53:26.649609 (+ 20us) server_negotiation.cc:407] Connection header received
0808 03:53:26.649849 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:53:26.649853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:53:26.649929 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:53:26.650056 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:53:26.650570 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.651627 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.653475 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.654127 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.654246 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:53:26.654927 (+ 681us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:53:26.655460 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:53:26.655622 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 03:53:26.655679 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 03:53:26.655759 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 03:53:26.666620 11858 raft_consensus.cc:1261] T 2d78be7fe57f423d8d02864f68a0ff08 P 6bd1ecdea73241039b18cfcdd2d19294 [term 1 FOLLOWER]: Refusing update from remote peer 9cf25e812601404e92d7250c60399b0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:53:26.666623 11440 raft_consensus.cc:1261] T 2d78be7fe57f423d8d02864f68a0ff08 P 6e3d3dbfb02b458dbf81a3966beabb29 [term 1 FOLLOWER]: Refusing update from remote peer 9cf25e812601404e92d7250c60399b0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:53:26.667879 11978 consensus_queue.cc:1035] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bd1ecdea73241039b18cfcdd2d19294" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 39677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:53:26.668362 11986 consensus_queue.cc:1035] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e3d3dbfb02b458dbf81a3966beabb29" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:53:26.682699 11992 mvcc.cc:205] Tried to move back new op lower bound from 7316116302494007296 to 7316116302211403776. Current Snapshot: MvccSnapshot[applied={T|T < 7316116302494007296}]
I20260808 03:53:26.704267 11183 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:53:26.704607 11183 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:53:26.709039 11183 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59805368628d4fd99017069684291f53: Sending DeleteTablet for 3 replicas of tablet 2d78be7fe57f423d8d02864f68a0ff08
I20260808 03:53:26.710268 11826 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d78be7fe57f423d8d02864f68a0ff08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:53:26 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:57062
I20260808 03:53:26.710383 11618 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d78be7fe57f423d8d02864f68a0ff08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:53:26 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:34060
I20260808 03:53:26.710373 11412 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d78be7fe57f423d8d02864f68a0ff08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:53:26 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:57574
I20260808 03:53:26.714848 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 11308
I20260808 03:53:26.722388 12007 tablet_replica.cc:330] T 2d78be7fe57f423d8d02864f68a0ff08 P 6e3d3dbfb02b458dbf81a3966beabb29: stopping tablet replica
I20260808 03:53:26.722874 12007 raft_consensus.cc:2229] T 2d78be7fe57f423d8d02864f68a0ff08 P 6e3d3dbfb02b458dbf81a3966beabb29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:53:26.723428 12007 raft_consensus.cc:2258] T 2d78be7fe57f423d8d02864f68a0ff08 P 6e3d3dbfb02b458dbf81a3966beabb29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:53:26.727095 12007 ts_tablet_manager.cc:1905] T 2d78be7fe57f423d8d02864f68a0ff08 P 6e3d3dbfb02b458dbf81a3966beabb29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:53:26.727376 12005 tablet_replica.cc:330] T 2d78be7fe57f423d8d02864f68a0ff08 P 6bd1ecdea73241039b18cfcdd2d19294: stopping tablet replica
I20260808 03:53:26.727888 12005 raft_consensus.cc:2229] T 2d78be7fe57f423d8d02864f68a0ff08 P 6bd1ecdea73241039b18cfcdd2d19294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:53:26.730099 12008 tablet_replica.cc:330] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f: stopping tablet replica
I20260808 03:53:26.730505 12008 raft_consensus.cc:2229] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:53:26.731029 12005 raft_consensus.cc:2258] T 2d78be7fe57f423d8d02864f68a0ff08 P 6bd1ecdea73241039b18cfcdd2d19294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:53:26.731385 12008 raft_consensus.cc:2258] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:53:26.733577 12008 ts_tablet_manager.cc:1905] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:53:26.735674 12007 ts_tablet_manager.cc:1918] T 2d78be7fe57f423d8d02864f68a0ff08 P 6e3d3dbfb02b458dbf81a3966beabb29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:53:26.736006 12007 log.cc:1196] T 2d78be7fe57f423d8d02864f68a0ff08 P 6e3d3dbfb02b458dbf81a3966beabb29: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-0/wal/wals/2d78be7fe57f423d8d02864f68a0ff08
Aug 08 03:53:26 dist-test-slave-jlzn krb5kdc[10479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.129@KRBTEST.COM
I20260808 03:53:26.736781 12005 ts_tablet_manager.cc:1905] T 2d78be7fe57f423d8d02864f68a0ff08 P 6bd1ecdea73241039b18cfcdd2d19294: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:53:26.736835 12007 ts_tablet_manager.cc:1939] T 2d78be7fe57f423d8d02864f68a0ff08 P 6e3d3dbfb02b458dbf81a3966beabb29: Deleting consensus metadata
I20260808 03:53:26.739141 11162 catalog_manager.cc:4841] TS 6e3d3dbfb02b458dbf81a3966beabb29 (127.26.146.129:32865): tablet 2d78be7fe57f423d8d02864f68a0ff08 (table test-table [id=4c55c7fed42948568328aebcaf4c5e4f]) successfully deleted
I20260808 03:53:26.741070 12008 ts_tablet_manager.cc:1918] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:53:26.741336 12008 log.cc:1196] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/2d78be7fe57f423d8d02864f68a0ff08
I20260808 03:53:26.742063 12008 ts_tablet_manager.cc:1939] T 2d78be7fe57f423d8d02864f68a0ff08 P 9cf25e812601404e92d7250c60399b0f: Deleting consensus metadata
I20260808 03:53:26.743924 11160 catalog_manager.cc:4841] TS 9cf25e812601404e92d7250c60399b0f (127.26.146.130:42443): tablet 2d78be7fe57f423d8d02864f68a0ff08 (table test-table [id=4c55c7fed42948568328aebcaf4c5e4f]) successfully deleted
I20260808 03:53:26.745819 12005 ts_tablet_manager.cc:1918] T 2d78be7fe57f423d8d02864f68a0ff08 P 6bd1ecdea73241039b18cfcdd2d19294: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:53:26.746045 12005 log.cc:1196] T 2d78be7fe57f423d8d02864f68a0ff08 P 6bd1ecdea73241039b18cfcdd2d19294: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/2d78be7fe57f423d8d02864f68a0ff08
I20260808 03:53:26.746717 12005 ts_tablet_manager.cc:1939] T 2d78be7fe57f423d8d02864f68a0ff08 P 6bd1ecdea73241039b18cfcdd2d19294: Deleting consensus metadata
I20260808 03:53:26.748632 11158 catalog_manager.cc:4841] TS 6bd1ecdea73241039b18cfcdd2d19294 (127.26.146.131:39677): tablet 2d78be7fe57f423d8d02864f68a0ff08 (table test-table [id=4c55c7fed42948568328aebcaf4c5e4f]) successfully deleted
I20260808 03:53:26.748696 11969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:53:26.716541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57586 (local address 127.26.146.129:32865)
0808 03:53:26.716744 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 03:53:26.716750 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:53:26.720054 (+ 3304us) server_negotiation.cc:407] Connection header received
0808 03:53:26.721138 (+ 1084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:53:26.721145 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:53:26.721272 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:53:26.721349 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:53:26.723869 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.725061 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.728131 (+ 3070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.729030 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.729135 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:53:26.738553 (+ 9418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:53:26.738569 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:53:26.738587 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:53:26.738629 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:53:26.744717 (+ 6088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:53:26.745751 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:53:26.745762 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:53:26.745766 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:53:26.746025 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:53:26.746538 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:53:26.746544 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:53:26.746547 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:53:26.747198 (+ 651us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:53:26.747657 (+ 459us) server_negotiation.cc:1035] Waiting for connection context
0808 03:53:26.747886 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 03:53:26.748121 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 03:53:26.749420 11496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:53:26.841115 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 11308
I20260808 03:53:26.866349 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 11511
Aug 08 03:53:26 dist-test-slave-jlzn krb5kdc[10479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.130@KRBTEST.COM
I20260808 03:53:26.885325 11963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:53:26.867314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34082 (local address 127.26.146.130:42443)
0808 03:53:26.867645 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0808 03:53:26.867652 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:53:26.868212 (+ 560us) server_negotiation.cc:407] Connection header received
0808 03:53:26.868515 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:53:26.868522 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:53:26.868644 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:53:26.868763 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:53:26.870544 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.871850 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.873615 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:26.874443 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:26.874614 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:53:26.879206 (+ 4592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:53:26.879220 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:53:26.879236 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:53:26.879281 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:53:26.882312 (+ 3031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:53:26.883048 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:53:26.883057 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:53:26.883060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:53:26.883233 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:53:26.883600 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:53:26.883605 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:53:26.883608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:53:26.884133 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:53:26.884424 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0808 03:53:26.884755 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 03:53:26.884984 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260808 03:53:26.886065 11712 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:53:27.000698 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 11511
I20260808 03:53:27.025137 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 11729
Aug 08 03:53:27 dist-test-slave-jlzn krb5kdc[10479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.131@KRBTEST.COM
I20260808 03:53:27.039990 11962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:53:27.025926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57076 (local address 127.26.146.131:39677)
0808 03:53:27.026089 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 03:53:27.026093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:53:27.026139 (+ 46us) server_negotiation.cc:407] Connection header received
0808 03:53:27.026310 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:53:27.026314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:53:27.026377 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:53:27.026460 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:53:27.027770 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:27.028492 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:27.029669 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:53:27.030047 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:53:27.030133 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:53:27.034490 (+ 4357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:53:27.034502 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:53:27.034519 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:53:27.034552 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:53:27.037257 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:53:27.037972 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:53:27.037980 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:53:27.037982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:53:27.038110 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:53:27.038473 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:53:27.038477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:53:27.038480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:53:27.038857 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:53:27.039142 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0808 03:53:27.039421 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 03:53:27.039624 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 03:53:27.040567 11927 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:53:27.128909 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 11729
I20260808 03:53:27.147512 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 11136
I20260808 03:53:27.148761 11226 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:53:27.258750 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 11136
I20260808 03:53:27.286669 27210 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 03:53:27.631503 27210 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 03:53:27.631769 27210 mini_ranger.cc:66] Stopping Ranger...
I20260808 03:53:27.976289 27210 mini_ranger.cc:68] Stopped Ranger
2026-08-08 03:53:27.976 UTC [10528] LOG: received smart shutdown request
2026-08-08 03:53:27.980 UTC [10528] LOG: background worker "logical replication launcher" (PID 10544) exited with exit code 1
2026-08-08 03:53:27.980 UTC [10536] LOG: shutting down
2026-08-08 03:53:28.116 UTC [10528] LOG: database system is shut down
2026-08-08T03:53:28Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49534 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:53:28 dist-test-slave-jlzn krb5kdc[12028](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:53:28 dist-test-slave-jlzn krb5kdc[12028](info): set up 2 sockets
Aug 08 03:53:28 dist-test-slave-jlzn krb5kdc[12028](info): commencing operation
krb5kdc: starting...
W20260808 03:53:29.428318 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.006s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:53:29 dist-test-slave-jlzn krb5kdc[12028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:53:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:53:29Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.26.146.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.26.146.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.26.146.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/rangeradmin_127.26.146.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.26.146.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/rangeradmin_127.26.146.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.26.146.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.26.146.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.26.146.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/rangerlookup_127.26.146.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.26.146.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/rangerlookup_127.26.146.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.146.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.146.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.146.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/HTTP_127.26.146.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.146.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/HTTP_127.26.146.148@KRBTEST.COM.keytab.
I20260808 03:53:29.787104 27210 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskT_CS6X/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/postgres -l logfile start
2026-08-08 03:53:32.111 UTC [12074] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 03:53:32.111 UTC [12074] LOG: listening on IPv4 address "127.26.146.148", port 47917
2026-08-08 03:53:32.115 UTC [12074] LOG: listening on Unix socket "/tmp/.s.PGSQL.47917"
2026-08-08 03:53:32.137 UTC [12077] LOG: database system was shut down at 2026-08-08 03:53:30 UTC
2026-08-08 03:53:32.140 UTC [12074] LOG: database system is ready to accept connections
I20260808 03:53:33.206983 27210 mini_postgres.cc:96] Postgres bound to 47917
2026-08-08 03:53:33.233 UTC [12087] FATAL: database "slave" does not exist
127.26.146.148:47917 - accepting connections
I20260808 03:53:33.233623 27210 mini_ranger.cc:161] Starting Ranger...
I20260808 03:53:33.262728 27210 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 03:53:33.670810 27210 mini_ranger.cc:87] Created ranger Postgres database
I20260808 03:53:33.670935 27210 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ranger-admin
2026-08-08 03:53:33,953 [I] DB FLAVOR :POSTGRES
2026-08-08 03:53:33,953 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 03:53:33,953 [I] Checking connection..
2026-08-08 03:53:33,953 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:47917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 03:53:34,208 [I] Checking connection passed.
2026-08-08 03:53:34,208 [I] --------- Verifying version history table ---------
2026-08-08 03:53:34,208 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:47917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 03:53:34,470 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 03:53:34,471 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:47917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 03:53:34,733 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 03:53:34,734 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 03:53:34,734 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:47917/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 03:53:34.984 UTC [12184] WARNING: there is no transaction in progress
2026-08-08 03:53:34,992 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 03:53:34,993 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:47917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 03:53:35,230 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 03:53:35,230 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 03:53:35,230 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:47917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 03:53:35,487 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:47917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 03:53:35,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:47917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-jlzn.c.gcp-upstream.internal','N') ;"
2026-08-08 03:53:35,977 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:47917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 03:53:36,235 [I] Table x_portal_user does not exist in database ranger
2026-08-08 03:53:36,236 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:47917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 03:53:36,475 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 03:53:36,475 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:47917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 03:53:36,706 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:47917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 03:53:36,963 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 03:53:36,963 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:47917/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 03:53:37.862 UTC [12346] WARNING: there is no transaction in progress
2026-08-08 03:53:37.878 UTC [12346] WARNING: there is no transaction in progress
2026-08-08 03:53:37.895 UTC [12346] WARNING: there is no transaction in progress
2026-08-08 03:53:37.910 UTC [12346] WARNING: there is no transaction in progress
2026-08-08 03:53:37.926 UTC [12346] WARNING: there is no transaction in progress
2026-08-08 03:53:37.942 UTC [12346] WARNING: there is no transaction in progress
2026-08-08 03:53:37.993 UTC [12346] WARNING: there is no transaction in progress
2026-08-08 03:53:38.000 UTC [12346] WARNING: there is no transaction in progress
2026-08-08 03:53:38.014 UTC [12346] WARNING: there is no transaction in progress
2026-08-08 03:53:38.025 UTC [12346] WARNING: there is no transaction in progress
2026-08-08 03:53:38.035 UTC [12346] WARNING: there is no transaction in progress
2026-08-08 03:53:38.049 UTC [12346] WARNING: there is no transaction in progress
2026-08-08 03:53:38.058 UTC [12346] WARNING: there is no transaction in progress
2026-08-08 03:53:38.064 UTC [12346] WARNING: there is no transaction in progress
2026-08-08 03:53:38.070 UTC [12346] WARNING: there is no transaction in progress
2026-08-08 03:53:38.076 UTC [12346] WARNING: there is no transaction in progress
2026-08-08 03:53:38,675 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 03:53:38,676 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:47917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 03:53:38,917 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 03:53:38,917 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:47917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 03:53:39,177 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 03:53:39,177 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:47917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 03:53:39,441 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 03:53:39,441 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:47917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 03:53:39,679 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:47917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 03:53:39,911 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:47917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-jlzn.c.gcp-upstream.internal';"
2026-08-08 03:53:40,164 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 03:53:40,165 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:47917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 03:53:40,427 [I] DB_PATCHES have already been applied
I20260808 03:53:40.432957 27210 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ranger-admin:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskT_CS6X/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 03:53:40.433046 27210 mini_ranger.cc:193] Using host: 127.26.146.148
I20260808 03:53:40.451684 27210 mini_ranger.cc:227] Ranger admin URL: http://127.26.146.148:46505
Aug 08, 2026 3:53:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 3:53:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 3:53:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 3:53:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 3:53:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 3:53:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 3:53:41 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 03:53:41.666558 27210 mini_ranger.cc:160] Time spent starting Ranger: real 8.433s user 0.008s sys 0.045s
Aug 08, 2026 3:54:01 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 3:54:01 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 3:54:01 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 3:54:01 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 3:54:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 3:54:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 3:54:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 3:54:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 3:54:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 3:54:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 3:54:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 3:54:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 3:54:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 3:54:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 3:54:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 3:54:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 3:54:01 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 3:54:02 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 03:54:02.991544 27210 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.26.146.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.26.146.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.26.146.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/rangerkms_127.26.146.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.26.146.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/rangerkms_127.26.146.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.146.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.146.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.146.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/HTTP_127.26.146.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.146.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/HTTP_127.26.146.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 03:54:03 dist-test-slave-jlzn krb5kdc[12028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161243, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 03:54:03.171830 27210 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 03:54:03.194701 27210 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 03:54:03.971689 27210 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 03:54:03.971849 27210 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ranger-kms
I20260808 03:54:03.971902 27210 mini_ranger_kms.cc:227] Using postgres at 127.26.146.148:47917
2026-08-08 03:54:04,307 [I] DB FLAVOR :POSTGRES
2026-08-08 03:54:04,307 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 03:54:04,307 [I] Checking connection
2026-08-08 03:54:04,307 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:47917/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 03:54:04,533 [I] connection success
2026-08-08 03:54:04,534 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 03:54:04,534 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 03:54:04,534 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:47917/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 03:54:04,790 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 03:54:04,790 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 03:54:04,790 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 03:54:04,790 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.146.148:47917/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 03:54:05,038 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 03:54:05.503708 27210 mini_ranger_kms.cc:310] Created kms service
I20260808 03:54:05.671671 27210 mini_ranger_kms.cc:326] Created kudu user
I20260808 03:54:05.714892 27210 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 03:54:05 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161243, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.26.146.148@KRBTEST.COM
I20260808 03:54:06.097266 27210 mini_ranger_kms.cc:384] Added ranger policy
I20260808 03:54:06.097607 27210 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ranger-kms:/tmp/dist-test-taskT_CS6X/build/asan/bin/postgresql.jar:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskT_CS6X/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 03:54:06.097666 27210 mini_ranger_kms.cc:240] Using host: 127.26.146.148
I20260808 03:54:06.124140 27210 mini_ranger_kms.cc:276] Ranger KMS PID: 12604
I20260808 03:54:06.124261 27210 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.26.146.148:51089
Aug 08, 2026 3:54:06 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 3:54:06 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 3:54:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 3:54:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 3:54:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 3:54:07 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 03:54:08 dist-test-slave-jlzn krb5kdc[12028](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786161248, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.146.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:08.604480 27210 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.433s user 0.070s sys 0.299s
I20260808 03:54:08.604754 27210 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 03:54:08.604914 27210 mini_ranger_kms.cc:401] 127.26.146.148:51089/kms/v1/keys
Aug 08 03:54:08 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161248, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.146.148@KRBTEST.COM for HTTP/127.26.146.148@KRBTEST.COM
Aug 08 03:54:08 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161248, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.146.148@KRBTEST.COM for HTTP/127.26.146.148@KRBTEST.COM
Aug 08 03:54:11 dist-test-slave-jlzn krb5kdc[12028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:54:11.335148 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:39185
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:39185
--ranger_config_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.146.148:51089/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:54:11.667680 12682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:11.667987 12682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:11.668119 12682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:11.675277 12682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:54:11.675382 12682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:11.675455 12682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:11.675534 12682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:54:11.675596 12682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:54:11.685444 12682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/wal
--ranger_kms_url=127.26.146.148:51089/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:39185
--ranger_config_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:39185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:11.686725 12682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:11.688035 12682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:11.699139 12690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:11.699167 12691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:11.699327 12693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:11.699905 12682 server_base.cc:1034] running on GCE node
I20260808 03:54:11.700636 12682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:11.701956 12682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:11.703147 12682 hybrid_clock.cc:648] HybridClock initialized: now 1786161251703093 us; error 45 us; skew 500 ppm
I20260808 03:54:11.703469 12682 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:11 dist-test-slave-jlzn krb5kdc[12028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161251, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:11.706413 12682 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:54:11.707696 12682 webserver.cc:460] Webserver started at http://127.26.146.190:38653/ using document root <none> and password file <none>
I20260808 03:54:11.708325 12682 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:11.708428 12682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:54:11.708724 12682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 03:54:11 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161251, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for HTTP/127.26.146.148@KRBTEST.COM
I20260808 03:54:11.760165 12682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "3c701d9c7d4645a09226eb2316dc1768"
format_stamp: "Formatted at 2026-08-08 03:54:11 on dist-test-slave-jlzn"
server_key: "f4faa9614ff8d1e59d96f10e5bd42dc2"
server_key_iv: "02031e830d7ba3648c1656da4a6e3a50"
server_key_version: "kuduclusterkey@0"
I20260808 03:54:11.761052 12682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "3c701d9c7d4645a09226eb2316dc1768"
format_stamp: "Formatted at 2026-08-08 03:54:11 on dist-test-slave-jlzn"
server_key: "f4faa9614ff8d1e59d96f10e5bd42dc2"
server_key_iv: "02031e830d7ba3648c1656da4a6e3a50"
server_key_version: "kuduclusterkey@0"
I20260808 03:54:11.766932 12682 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 03:54:11.787096 12702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:11.788501 12682 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 03:54:11.788679 12682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "3c701d9c7d4645a09226eb2316dc1768"
format_stamp: "Formatted at 2026-08-08 03:54:11 on dist-test-slave-jlzn"
server_key: "f4faa9614ff8d1e59d96f10e5bd42dc2"
server_key_iv: "02031e830d7ba3648c1656da4a6e3a50"
server_key_version: "kuduclusterkey@0"
I20260808 03:54:11.788894 12682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:11.831734 12682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:11.835681 12682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:11.836009 12682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:11.853223 12682 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:39185
I20260808 03:54:11.853247 12773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:39185 every 8 connection(s)
I20260808 03:54:11.854543 12682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:54:11.858217 12775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:54:11.864934 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 12682
I20260808 03:54:11.865190 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:54:11.866008 27210 external_mini_cluster.cc:1428] Setting key ded0834b65d2fbcfb7bcdb2471fe07e8
I20260808 03:54:11.869107 12775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768: Bootstrap starting.
I20260808 03:54:11.872447 12775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768: Neither blocks nor log segments found. Creating new log.
I20260808 03:54:11.873601 12775 log.cc:824] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768: Log is configured to *not* fsync() on all Append() calls
I20260808 03:54:11.876876 12775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768: No bootstrap required, opened a new log
Aug 08 03:54:11 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:54:11.885478 12775 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c701d9c7d4645a09226eb2316dc1768" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 39185 } }
I20260808 03:54:11.885854 12775 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:54:11.885951 12775 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c701d9c7d4645a09226eb2316dc1768, State: Initialized, Role: FOLLOWER
I20260808 03:54:11.886560 12775 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c701d9c7d4645a09226eb2316dc1768" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 39185 } }
I20260808 03:54:11.886813 12775 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:54:11.886950 12775 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:54:11.887117 12775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:54:11.889238 12778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:11.868232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37720 (local address 127.26.146.190:39185)
0808 03:54:11.868983 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:11.868994 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:11.869035 (+ 41us) server_negotiation.cc:407] Connection header received
0808 03:54:11.870742 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:11.870765 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:11.871118 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:11.871408 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:11.873101 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:11.874623 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:11.876467 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:11.877152 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:11.877287 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:11.881661 (+ 4374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:11.881669 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:11.881686 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:11.881729 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:11.885191 (+ 3462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:11.885850 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:11.885858 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:11.885867 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:11.886050 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:11.886416 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:11.886421 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:11.886425 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:11.887050 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:11.887358 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:11.887847 (+ 489us) server_negotiation.cc:299] Negotiation successful
0808 03:54:11.888249 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":333,"threads_started":1}
I20260808 03:54:11.889600 12775 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c701d9c7d4645a09226eb2316dc1768" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 39185 } }
I20260808 03:54:11.890095 12775 leader_election.cc:302] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c701d9c7d4645a09226eb2316dc1768; no voters:
I20260808 03:54:11.890763 12775 leader_election.cc:288] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:54:11.890887 12780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:54:11.892575 12780 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [term 1 LEADER]: Becoming Leader. State: Replica: 3c701d9c7d4645a09226eb2316dc1768, State: Running, Role: LEADER
I20260808 03:54:11.893051 12775 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:54:11.893105 12780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c701d9c7d4645a09226eb2316dc1768" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 39185 } }
I20260808 03:54:11.896653 12782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c701d9c7d4645a09226eb2316dc1768" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c701d9c7d4645a09226eb2316dc1768" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 39185 } } }
I20260808 03:54:11.896731 12783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c701d9c7d4645a09226eb2316dc1768. Latest consensus state: current_term: 1 leader_uuid: "3c701d9c7d4645a09226eb2316dc1768" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c701d9c7d4645a09226eb2316dc1768" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 39185 } } }
I20260808 03:54:11.897038 12782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [sys.catalog]: This master's current role is: LEADER
I20260808 03:54:11.897066 12783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [sys.catalog]: This master's current role is: LEADER
I20260808 03:54:11.897886 12793 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:54:11.904508 12793 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:54:11.905256 12775 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 03:54:11.922958 12793 catalog_manager.cc:1347] Generated new cluster ID: 9c2cd106eb11447e817b3851ca5c5f24
I20260808 03:54:11.923105 12793 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:54:11.937064 12793 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:54:11.939005 12793 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:54:11.954675 12793 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768: Generated new TSK 0
I20260808 03:54:11.955760 12793 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
I20260808 03:54:12.723441 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.146.148:51089/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:39185
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 03:54:12 dist-test-slave-jlzn krb5kdc[12028](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786161252, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 03:54:13.123312 12851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:13.123776 12851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:13.124006 12851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:13.135607 12851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:13.135747 12851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:13.136003 12851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:54:13.151723 12851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.26.146.148:51089/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:39185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:13.153502 12851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:13.155272 12851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:13.180832 12870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:13.188818 12871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:13.190920 12874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:13.192008 12851 server_base.cc:1034] running on GCE node
I20260808 03:54:13.193184 12851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:13.194664 12851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:13.195927 12851 hybrid_clock.cc:648] HybridClock initialized: now 1786161253195857 us; error 55 us; skew 500 ppm
I20260808 03:54:13.196301 12851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:13 dist-test-slave-jlzn krb5kdc[12028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161253, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:13.202591 12851 init.cc:377] Logged in from keytab as kudu/127.26.146.129@KRBTEST.COM (short username kudu)
I20260808 03:54:13.204584 12851 webserver.cc:460] Webserver started at http://127.26.146.129:37909/ using document root <none> and password file <none>
I20260808 03:54:13.205569 12851 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:13.205718 12851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:54:13.206106 12851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 03:54:13 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161253, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for HTTP/127.26.146.148@KRBTEST.COM
I20260808 03:54:13.236922 12851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "4cd8edd17c354aaa9601089057103013"
format_stamp: "Formatted at 2026-08-08 03:54:13 on dist-test-slave-jlzn"
server_key: "2352b1b5ebd39285aced5c515b85caee"
server_key_iv: "c69d14f8d46949db3f360530f5982dcc"
server_key_version: "kuduclusterkey@0"
I20260808 03:54:13.238440 12851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "4cd8edd17c354aaa9601089057103013"
format_stamp: "Formatted at 2026-08-08 03:54:13 on dist-test-slave-jlzn"
server_key: "2352b1b5ebd39285aced5c515b85caee"
server_key_iv: "c69d14f8d46949db3f360530f5982dcc"
server_key_version: "kuduclusterkey@0"
I20260808 03:54:13.247138 12851 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.008s sys 0.000s
I20260808 03:54:13.273365 12883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:13.275430 12851 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:54:13.275671 12851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "4cd8edd17c354aaa9601089057103013"
format_stamp: "Formatted at 2026-08-08 03:54:13 on dist-test-slave-jlzn"
server_key: "2352b1b5ebd39285aced5c515b85caee"
server_key_iv: "c69d14f8d46949db3f360530f5982dcc"
server_key_version: "kuduclusterkey@0"
I20260808 03:54:13.276026 12851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:13.305836 12851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:13.311129 12851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:13.311631 12851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:13.312968 12851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:54:13.322962 12851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:54:13.323143 12851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260808 03:54:13.323287 12851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:54:13.323345 12851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 03:54:13 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161252, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for HTTP/127.26.146.148@KRBTEST.COM
I20260808 03:54:13.516059 12851 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:43221
I20260808 03:54:13.516103 13062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:43221 every 8 connection(s)
I20260808 03:54:13.517753 12851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:54:13.523838 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 12851
I20260808 03:54:13.524086 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:54:13.524955 27210 external_mini_cluster.cc:1428] Setting key 09789b9fc1f9b8af86c7767b71afe0c4
Aug 08 03:54:13 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161253, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:54:13.543913 13069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:13.520773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:44289 (local address 127.26.146.190:39185)
0808 03:54:13.521277 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:13.521284 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:13.522351 (+ 1067us) server_negotiation.cc:407] Connection header received
0808 03:54:13.525015 (+ 2664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:13.525022 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:13.525118 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:13.525200 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:13.527307 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:13.528478 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:13.531541 (+ 3063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:13.532228 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:13.532340 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:13.536659 (+ 4319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:13.536666 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:13.536671 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:13.536712 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:13.540267 (+ 3555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:13.541177 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:13.541183 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:13.541187 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:13.541346 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:13.541847 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:13.541852 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:13.541855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:13.542219 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:13.542357 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:13.543263 (+ 906us) server_negotiation.cc:299] Negotiation successful
0808 03:54:13.543488 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":246,"threads_started":1}
I20260808 03:54:13.545152 13070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:13.521310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:39185 (local address 127.26.146.129:44289)
0808 03:54:13.522186 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:13.522243 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:13.524690 (+ 2447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:13.525738 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:13.525749 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:13.526340 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:13.527111 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:13.527136 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:13.529361 (+ 2225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:13.529367 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:13.531346 (+ 1979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:13.531362 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:13.532687 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:13.532694 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:13.532964 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:13.533768 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:13.533798 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:13.536398 (+ 2600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:13.540419 (+ 4021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:13.540428 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:13.540447 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:13.541048 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:13.541484 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:13.541490 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:13.541493 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:13.541742 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:13.542363 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:13.542372 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:13.542815 (+ 443us) client_negotiation.cc:769] Sending connection context
0808 03:54:13.543156 (+ 341us) client_negotiation.cc:240] Negotiation successful
0808 03:54:13.543707 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":354,"threads_started":1}
I20260808 03:54:13.546936 13065 heartbeater.cc:347] Connected to a master server at 127.26.146.190:39185
I20260808 03:54:13.547284 13065 heartbeater.cc:464] Registering TS with master...
I20260808 03:54:13.548153 13065 heartbeater.cc:511] Master 127.26.146.190:39185 requested a full tablet report, sending...
I20260808 03:54:13.550652 12723 ts_manager.cc:194] Registered new tserver with Master: 4cd8edd17c354aaa9601089057103013 (127.26.146.129:43221)
I20260808 03:54:13.552735 12723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.129@KRBTEST.COM'} at 127.26.146.129:44289
WARNING: no policy specified for kudu/127.26.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
I20260808 03:54:13.611045 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.146.148:51089/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:39185
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 03:54:13 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161252, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for HTTP/127.26.146.148@KRBTEST.COM
W20260808 03:54:14.079579 13076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:14.079881 13076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:14.080020 13076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:14.087112 13076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:14.087227 13076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:14.087375 13076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:54:14.097129 13076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.26.146.148:51089/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:39185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:14.098376 13076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:14.099732 13076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:14.112612 13097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:14.112581 13098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:14.113816 13102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:14.114933 13076 server_base.cc:1034] running on GCE node
I20260808 03:54:14.115653 13076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:14.116895 13076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:14.118151 13076 hybrid_clock.cc:648] HybridClock initialized: now 1786161254118057 us; error 77 us; skew 500 ppm
I20260808 03:54:14.118511 13076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:14 dist-test-slave-jlzn krb5kdc[12028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161254, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:14.122521 13076 init.cc:377] Logged in from keytab as kudu/127.26.146.130@KRBTEST.COM (short username kudu)
I20260808 03:54:14.124353 13076 webserver.cc:460] Webserver started at http://127.26.146.130:40615/ using document root <none> and password file <none>
I20260808 03:54:14.125335 13076 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:14.125478 13076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:54:14.125842 13076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 03:54:14 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161254, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for HTTP/127.26.146.148@KRBTEST.COM
I20260808 03:54:14.142165 13076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "8d459f0b78c9482a8adc2f03989bd42c"
format_stamp: "Formatted at 2026-08-08 03:54:14 on dist-test-slave-jlzn"
server_key: "f0da3107a0fa0f887a5e997cbc2e2c74"
server_key_iv: "2a9265ae134313a14494e14649808573"
server_key_version: "kuduclusterkey@0"
I20260808 03:54:14.143203 13076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "8d459f0b78c9482a8adc2f03989bd42c"
format_stamp: "Formatted at 2026-08-08 03:54:14 on dist-test-slave-jlzn"
server_key: "f0da3107a0fa0f887a5e997cbc2e2c74"
server_key_iv: "2a9265ae134313a14494e14649808573"
server_key_version: "kuduclusterkey@0"
I20260808 03:54:14.149396 13076 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260808 03:54:14.166726 13110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:14.168347 13076 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.002s
I20260808 03:54:14.168516 13076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "8d459f0b78c9482a8adc2f03989bd42c"
format_stamp: "Formatted at 2026-08-08 03:54:14 on dist-test-slave-jlzn"
server_key: "f0da3107a0fa0f887a5e997cbc2e2c74"
server_key_iv: "2a9265ae134313a14494e14649808573"
server_key_version: "kuduclusterkey@0"
I20260808 03:54:14.168720 13076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:14.189798 13076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:14.194000 13076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:14.194303 13076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:14.195307 13076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:54:14.198271 13076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:54:14.198354 13076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:14.198473 13076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:54:14.198524 13076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:14.234156 13076 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:43063
I20260808 03:54:14.234234 13273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:43063 every 8 connection(s)
I20260808 03:54:14.235423 13076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:54:14.239539 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 13076
I20260808 03:54:14.239722 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:54:14.240372 27210 external_mini_cluster.cc:1428] Setting key daf01b2d8ad025a25074b3569604065e
Aug 08 03:54:14 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161254, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:54:14.259662 13278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:14.238292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:51069 (local address 127.26.146.190:39185)
0808 03:54:14.238871 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:14.238877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:14.241934 (+ 3057us) server_negotiation.cc:407] Connection header received
0808 03:54:14.244510 (+ 2576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:14.244516 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:14.244596 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:14.244701 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:14.246663 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:14.247748 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:14.249842 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:14.250466 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:14.250589 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:14.254194 (+ 3605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:14.254199 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:14.254202 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:14.254233 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:14.256784 (+ 2551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:14.257448 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:14.257453 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:14.257455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:14.257581 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:14.257991 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:14.257996 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:14.257999 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:14.258271 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:14.258401 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:14.259061 (+ 660us) server_negotiation.cc:299] Negotiation successful
0808 03:54:14.259294 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":318,"threads_started":1}
I20260808 03:54:14.260797 13279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:14.238765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:39185 (local address 127.26.146.130:51069)
0808 03:54:14.241755 (+ 2990us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:14.241813 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:14.244097 (+ 2284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:14.245134 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:14.245145 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:14.245698 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:14.246466 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:14.246491 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:14.247941 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:14.247946 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:14.249685 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:14.249700 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:14.250700 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:14.250707 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:14.250963 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:14.251715 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:14.251744 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:14.253988 (+ 2244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:14.256929 (+ 2941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:14.256938 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:14.256954 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:14.257338 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:14.257681 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:14.257687 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:14.257691 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:14.257886 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:14.258392 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:14.258399 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:14.258711 (+ 312us) client_negotiation.cc:769] Sending connection context
0808 03:54:14.258940 (+ 229us) client_negotiation.cc:240] Negotiation successful
0808 03:54:14.259426 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2398,"thread_start_us":2175,"threads_started":1}
I20260808 03:54:14.262115 13274 heartbeater.cc:347] Connected to a master server at 127.26.146.190:39185
I20260808 03:54:14.262395 13274 heartbeater.cc:464] Registering TS with master...
I20260808 03:54:14.263119 13274 heartbeater.cc:511] Master 127.26.146.190:39185 requested a full tablet report, sending...
I20260808 03:54:14.264599 12723 ts_manager.cc:194] Registered new tserver with Master: 8d459f0b78c9482a8adc2f03989bd42c (127.26.146.130:43063)
I20260808 03:54:14.265381 12723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.130@KRBTEST.COM'} at 127.26.146.130:51069
WARNING: no policy specified for kudu/127.26.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
I20260808 03:54:14.311657 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.146.148:51089/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:39185
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:54:14.557497 13065 heartbeater.cc:503] Master 127.26.146.190:39185 was elected leader, sending a full tablet report...
W20260808 03:54:14.744211 13284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:14.744645 13284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:14.744844 13284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:14.753149 13284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:14.753355 13284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:14.753582 13284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:54:14.764775 13284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.26.146.148:51089/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:39185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:14.766186 13284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:14.767642 13284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:14.780344 13293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:14.780341 13291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:14.780623 13297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:14.781509 13284 server_base.cc:1034] running on GCE node
I20260808 03:54:14.782068 13284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:14.782943 13284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:14.784145 13284 hybrid_clock.cc:648] HybridClock initialized: now 1786161254784090 us; error 45 us; skew 500 ppm
I20260808 03:54:14.784432 13284 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:14 dist-test-slave-jlzn krb5kdc[12028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161254, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:14.787405 13284 init.cc:377] Logged in from keytab as kudu/127.26.146.131@KRBTEST.COM (short username kudu)
I20260808 03:54:14.788802 13284 webserver.cc:460] Webserver started at http://127.26.146.131:43955/ using document root <none> and password file <none>
I20260808 03:54:14.789450 13284 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:14.789568 13284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:54:14.789853 13284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 03:54:14 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161254, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for HTTP/127.26.146.148@KRBTEST.COM
I20260808 03:54:14.803110 13284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "b9dbadd80a5542cbb9483b1cb86fcd51"
format_stamp: "Formatted at 2026-08-08 03:54:14 on dist-test-slave-jlzn"
server_key: "8300bd66aebd5a79395c7d4daeb15d01"
server_key_iv: "2f394107d356726eb5a3a9be5bb573eb"
server_key_version: "kuduclusterkey@0"
I20260808 03:54:14.803973 13284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "b9dbadd80a5542cbb9483b1cb86fcd51"
format_stamp: "Formatted at 2026-08-08 03:54:14 on dist-test-slave-jlzn"
server_key: "8300bd66aebd5a79395c7d4daeb15d01"
server_key_iv: "2f394107d356726eb5a3a9be5bb573eb"
server_key_version: "kuduclusterkey@0"
I20260808 03:54:14.809517 13284 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 03:54:14.824229 13305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:14.825675 13284 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 03:54:14.825850 13284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "b9dbadd80a5542cbb9483b1cb86fcd51"
format_stamp: "Formatted at 2026-08-08 03:54:14 on dist-test-slave-jlzn"
server_key: "8300bd66aebd5a79395c7d4daeb15d01"
server_key_iv: "2f394107d356726eb5a3a9be5bb573eb"
server_key_version: "kuduclusterkey@0"
I20260808 03:54:14.826061 13284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:14.843966 13284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:14.847482 13284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:14.847759 13284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:14.848666 13284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:54:14.849968 13284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:54:14.850046 13284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:14.850168 13284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:54:14.850225 13284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:14.885771 13284 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:45327
I20260808 03:54:14.885813 13468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:45327 every 8 connection(s)
I20260808 03:54:14.887012 13284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:54:14.891788 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 13284
I20260808 03:54:14.891986 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:54:14.892582 27210 external_mini_cluster.cc:1428] Setting key a92a974c8497705313765767849b772b
Aug 08 03:54:14 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161254, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:54:14.906126 13473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:14.889527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:41135 (local address 127.26.146.190:39185)
0808 03:54:14.890003 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:14.890008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:14.890891 (+ 883us) server_negotiation.cc:407] Connection header received
0808 03:54:14.893293 (+ 2402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:14.893299 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:14.893408 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:14.893535 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:14.895223 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:14.896096 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:14.897587 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:14.898022 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:14.898109 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:14.901267 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:14.901272 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:14.901275 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:14.901331 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:14.903563 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:14.904244 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:14.904248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:14.904250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:14.904380 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:14.904738 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:14.904741 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:14.904743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:14.904992 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:14.905109 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:14.905698 (+ 589us) server_negotiation.cc:299] Negotiation successful
0808 03:54:14.905861 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":236,"threads_started":1}
I20260808 03:54:14.907083 13477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:14.890059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:39185 (local address 127.26.146.131:41135)
0808 03:54:14.890736 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:14.890780 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:14.892981 (+ 2201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:14.893774 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:14.893784 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:14.894314 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:14.895029 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:14.895058 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:14.896233 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:14.896236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:14.897464 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:14.897486 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:14.898181 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:14.898186 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:14.898372 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:14.898947 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:14.898969 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:14.901059 (+ 2090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:14.903679 (+ 2620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:14.903685 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:14.903697 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:14.904138 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:14.904463 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:14.904466 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:14.904468 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:14.904652 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:14.905091 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:14.905097 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:14.905385 (+ 288us) client_negotiation.cc:769] Sending connection context
0808 03:54:14.905591 (+ 206us) client_negotiation.cc:240] Negotiation successful
0808 03:54:14.905931 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":265,"threads_started":1}
I20260808 03:54:14.908432 13469 heartbeater.cc:347] Connected to a master server at 127.26.146.190:39185
I20260808 03:54:14.908684 13469 heartbeater.cc:464] Registering TS with master...
I20260808 03:54:14.909343 13469 heartbeater.cc:511] Master 127.26.146.190:39185 requested a full tablet report, sending...
I20260808 03:54:14.910610 12722 ts_manager.cc:194] Registered new tserver with Master: b9dbadd80a5542cbb9483b1cb86fcd51 (127.26.146.131:45327)
I20260808 03:54:14.911468 12722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.131@KRBTEST.COM'} at 127.26.146.131:41135
I20260808 03:54:14.920022 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:54:14.921612 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 12851
Aug 08 03:54:14 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.129@KRBTEST.COM
I20260808 03:54:14.937141 13479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:14.922549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42320 (local address 127.26.146.129:43221)
0808 03:54:14.923112 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:14.923118 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:14.923140 (+ 22us) server_negotiation.cc:407] Connection header received
0808 03:54:14.923192 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:14.923197 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:14.923381 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:14.923495 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:14.924977 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:14.925978 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:14.927227 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:14.927746 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:14.927874 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:14.932021 (+ 4147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:14.932032 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:14.932046 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:14.932084 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:14.934668 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:14.935290 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:14.935296 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:14.935299 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:14.935463 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:14.935760 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:14.935763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:14.935765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:14.936124 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:14.936347 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:14.936675 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 03:54:14.936876 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":281,"threads_started":1}
I20260808 03:54:14.938414 13055 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:15.019217 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 12851
I20260808 03:54:15.040326 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 13076
Aug 08 03:54:15 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.130@KRBTEST.COM
I20260808 03:54:15.056314 13481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:15.041204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45398 (local address 127.26.146.130:43063)
0808 03:54:15.041768 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:15.041775 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:15.041799 (+ 24us) server_negotiation.cc:407] Connection header received
0808 03:54:15.041860 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:15.041867 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:15.042108 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:15.042256 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:15.043726 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:15.044770 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:15.046245 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:15.046795 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:15.046911 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:15.051254 (+ 4343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:15.051266 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:15.051282 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:15.051318 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:15.053959 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:15.054549 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:15.054557 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:15.054559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:15.054685 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:15.054958 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:15.054961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:15.054963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:15.055308 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:15.055539 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:15.055776 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 03:54:15.056017 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":287,"threads_started":1}
I20260808 03:54:15.057572 13268 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:15.142656 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 13076
I20260808 03:54:15.159490 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 13284
Aug 08 03:54:15 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.131@KRBTEST.COM
I20260808 03:54:15.176059 13484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:15.160531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40258 (local address 127.26.146.131:45327)
0808 03:54:15.161179 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:15.161186 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:15.161215 (+ 29us) server_negotiation.cc:407] Connection header received
0808 03:54:15.161275 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:15.161281 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:15.161490 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:15.161649 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:15.162994 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:15.164186 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:15.165484 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:15.166070 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:15.166190 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:15.170628 (+ 4438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:15.170639 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:15.170653 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:15.170690 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:15.173344 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:15.173970 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:15.173976 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:15.173979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:15.174120 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:15.174466 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:15.174470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:15.174472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:15.174811 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:15.175116 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:15.175418 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 03:54:15.175685 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":344,"threads_started":1}
I20260808 03:54:15.177343 13461 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:15.259932 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 13284
I20260808 03:54:15.279918 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 12682
I20260808 03:54:15.281160 12765 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:15.386148 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 12682
I20260808 03:54:15.414753 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:39185
--webserver_interface=127.26.146.190
--webserver_port=38653
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:39185
--ranger_config_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.146.148:51089/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.146.148:51089/kms with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:54:15.736835 13489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:15.737159 13489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:15.737300 13489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:15.744520 13489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:54:15.744642 13489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:15.744724 13489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:15.744803 13489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:54:15.744868 13489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:54:15.755137 13489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.146.148:51089/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:39185
--ranger_config_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:39185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=38653
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:15.756457 13489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:15.757797 13489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:15.769647 13497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:15.769694 13499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:15.769722 13494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:15.770535 13489 server_base.cc:1034] running on GCE node
I20260808 03:54:15.771260 13489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:15.772680 13489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:15.773903 13489 hybrid_clock.cc:648] HybridClock initialized: now 1786161255773852 us; error 39 us; skew 500 ppm
I20260808 03:54:15.774227 13489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:15 dist-test-slave-jlzn krb5kdc[12028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161255, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:15.777259 13489 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:54:15.778743 13489 webserver.cc:460] Webserver started at http://127.26.146.190:38653/ using document root <none> and password file <none>
I20260808 03:54:15.779328 13489 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:15.779435 13489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 03:54:15 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161255, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for HTTP/127.26.146.148@KRBTEST.COM
I20260808 03:54:15.855450 13489 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 03:54:15.858494 13511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:15.859895 13489 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:54:15.860086 13489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "3c701d9c7d4645a09226eb2316dc1768"
format_stamp: "Formatted at 2026-08-08 03:54:11 on dist-test-slave-jlzn"
server_key: "f4faa9614ff8d1e59d96f10e5bd42dc2"
server_key_iv: "02031e830d7ba3648c1656da4a6e3a50"
server_key_version: "kuduclusterkey@0"
I20260808 03:54:15.861032 13489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:15.876287 13489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:15.879843 13489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:15.880119 13489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:15.897157 13489 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:39185
I20260808 03:54:15.897178 13578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:39185 every 8 connection(s)
I20260808 03:54:15.898605 13489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:54:15.900652 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 13489
I20260808 03:54:15.901472 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.146.148:51089/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:43221
--local_ip_for_outbound_sockets=127.26.146.129
--tserver_master_addrs=127.26.146.190:39185
--webserver_port=37909
--webserver_interface=127.26.146.129
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.146.148:51089/kms with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:54:15.905349 13580 sys_catalog.cc:263] Verifying existing consensus state
I20260808 03:54:15.908488 13580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768: Bootstrap starting.
I20260808 03:54:15.926314 13580 log.cc:824] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768: Log is configured to *not* fsync() on all Append() calls
I20260808 03:54:15.934995 13580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 03:54:15.935487 13580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768: Bootstrap complete.
I20260808 03:54:15.944137 13580 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c701d9c7d4645a09226eb2316dc1768" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 39185 } }
I20260808 03:54:15.944964 13580 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c701d9c7d4645a09226eb2316dc1768, State: Initialized, Role: FOLLOWER
I20260808 03:54:15.945550 13580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c701d9c7d4645a09226eb2316dc1768" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 39185 } }
I20260808 03:54:15.945775 13580 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:54:15.945889 13580 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:54:15.946051 13580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [term 1 FOLLOWER]: Advancing to term 2
I20260808 03:54:15.949762 13580 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c701d9c7d4645a09226eb2316dc1768" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 39185 } }
I20260808 03:54:15.950220 13580 leader_election.cc:302] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c701d9c7d4645a09226eb2316dc1768; no voters:
I20260808 03:54:15.950798 13580 leader_election.cc:288] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 03:54:15.950937 13588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 03:54:15.951990 13588 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [term 2 LEADER]: Becoming Leader. State: Replica: 3c701d9c7d4645a09226eb2316dc1768, State: Running, Role: LEADER
I20260808 03:54:15.952682 13588 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c701d9c7d4645a09226eb2316dc1768" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 39185 } }
I20260808 03:54:15.952986 13580 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:54:15.955313 13589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3c701d9c7d4645a09226eb2316dc1768" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c701d9c7d4645a09226eb2316dc1768" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 39185 } } }
I20260808 03:54:15.956018 13589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [sys.catalog]: This master's current role is: LEADER
I20260808 03:54:15.955721 13590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c701d9c7d4645a09226eb2316dc1768. Latest consensus state: current_term: 2 leader_uuid: "3c701d9c7d4645a09226eb2316dc1768" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c701d9c7d4645a09226eb2316dc1768" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 39185 } } }
I20260808 03:54:15.956557 13590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768 [sys.catalog]: This master's current role is: LEADER
I20260808 03:54:15.966111 13597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:54:15.971971 13597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:54:15.973448 13597 catalog_manager.cc:1259] Loaded cluster ID: 9c2cd106eb11447e817b3851ca5c5f24
I20260808 03:54:15.973616 13597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:54:15.977763 13597 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:54:15.979332 13597 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768: Loaded TSK: 0
I20260808 03:54:15.980948 13597 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:54:15.990710 13580 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260808 03:54:16.359314 13584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:16.359887 13584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:16.360106 13584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:16.371392 13584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:16.371680 13584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:16.371979 13584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:54:16.387490 13584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.146.148:51089/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:43221
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=37909
--tserver_master_addrs=127.26.146.190:39185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:16.389616 13584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:16.391539 13584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:16.413728 13635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:16.416821 13637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:16.418419 13640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:16.419585 13584 server_base.cc:1034] running on GCE node
I20260808 03:54:16.420550 13584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:16.421603 13584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:16.422894 13584 hybrid_clock.cc:648] HybridClock initialized: now 1786161256422823 us; error 48 us; skew 500 ppm
I20260808 03:54:16.423288 13584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:16 dist-test-slave-jlzn krb5kdc[12028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161256, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:16.427450 13584 init.cc:377] Logged in from keytab as kudu/127.26.146.129@KRBTEST.COM (short username kudu)
I20260808 03:54:16.429442 13584 webserver.cc:460] Webserver started at http://127.26.146.129:37909/ using document root <none> and password file <none>
I20260808 03:54:16.430408 13584 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:16.430598 13584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 03:54:16 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161256, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for HTTP/127.26.146.148@KRBTEST.COM
I20260808 03:54:16.469851 13584 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.007s sys 0.003s
I20260808 03:54:16.479732 13650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:16.481729 13584 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.006s sys 0.005s
I20260808 03:54:16.482007 13584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "4cd8edd17c354aaa9601089057103013"
format_stamp: "Formatted at 2026-08-08 03:54:13 on dist-test-slave-jlzn"
server_key: "2352b1b5ebd39285aced5c515b85caee"
server_key_iv: "c69d14f8d46949db3f360530f5982dcc"
server_key_version: "kuduclusterkey@0"
I20260808 03:54:16.483536 13584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:16.516151 13584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:16.520825 13584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:16.521183 13584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:16.522285 13584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:54:16.523689 13584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:54:16.523794 13584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:16.523929 13584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:54:16.524006 13584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:16.612531 13584 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:43221
I20260808 03:54:16.612581 13819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:43221 every 8 connection(s)
I20260808 03:54:16.614179 13584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:54:16.624576 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 13584
I20260808 03:54:16.625442 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.146.148:51089/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:43063
--local_ip_for_outbound_sockets=127.26.146.130
--tserver_master_addrs=127.26.146.190:39185
--webserver_port=40615
--webserver_interface=127.26.146.130
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.146.148:51089/kms with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 03:54:16 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161256, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:54:16.657859 13824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:16.617440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:39185 (local address 127.26.146.129:39319)
0808 03:54:16.618226 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:16.618275 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:16.620836 (+ 2561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:16.637928 (+ 17092us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:16.637941 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:16.638635 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:16.639452 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:16.639483 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:16.641208 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:16.641212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:16.642708 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:16.642722 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:16.644001 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:16.644008 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:16.644293 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:16.645093 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:16.645123 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:16.648327 (+ 3204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:16.652676 (+ 4349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:16.652686 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:16.652701 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:16.653332 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:16.653877 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:16.653882 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:16.653885 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:16.654146 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:16.655252 (+ 1106us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:16.655261 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:16.655672 (+ 411us) client_negotiation.cc:769] Sending connection context
0808 03:54:16.656019 (+ 347us) client_negotiation.cc:240] Negotiation successful
0808 03:54:16.656570 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":1254,"threads_started":1}
I20260808 03:54:16.659907 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:16.618476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:39319 (local address 127.26.146.190:39185)
0808 03:54:16.636208 (+ 17732us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:16.636226 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:16.636277 (+ 51us) server_negotiation.cc:407] Connection header received
0808 03:54:16.636670 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:16.636687 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:16.637119 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:16.637640 (+ 521us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:16.639727 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:16.641019 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:16.642861 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:16.643752 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:16.643980 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:16.648679 (+ 4699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:16.648691 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:16.648713 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:16.648763 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:16.652485 (+ 3722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:16.653506 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:16.653516 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:16.653524 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:16.653731 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:16.654326 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:16.654331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:16.654335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:16.654998 (+ 663us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:16.655170 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:16.658755 (+ 3585us) server_negotiation.cc:299] Negotiation successful
0808 03:54:16.659045 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17455,"thread_start_us":5319,"threads_started":1}
I20260808 03:54:16.662546 13820 heartbeater.cc:347] Connected to a master server at 127.26.146.190:39185
I20260808 03:54:16.662842 13820 heartbeater.cc:464] Registering TS with master...
I20260808 03:54:16.663551 13820 heartbeater.cc:511] Master 127.26.146.190:39185 requested a full tablet report, sending...
W20260808 03:54:16.666523 13820 heartbeater.cc:644] Failed to heartbeat to 127.26.146.190:39185 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 03:54:16.666742 13820 heartbeater.cc:464] Registering TS with master...
I20260808 03:54:16.666983 13820 heartbeater.cc:511] Master 127.26.146.190:39185 requested a full tablet report, sending...
I20260808 03:54:16.668026 13820 heartbeater.cc:464] Registering TS with master...
I20260808 03:54:16.668259 13820 heartbeater.cc:511] Master 127.26.146.190:39185 requested a full tablet report, sending...
W20260808 03:54:16.669211 13820 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 03:54:17.049127 13826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:17.049526 13826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:17.049691 13826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:17.060976 13826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:17.061127 13826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:17.061321 13826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:54:17.076715 13826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.146.148:51089/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:43063
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=40615
--tserver_master_addrs=127.26.146.190:39185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:17.078428 13826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:17.080277 13826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:17.097306 13847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:17.104149 13848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:17.105656 13851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:17.111757 13826 server_base.cc:1034] running on GCE node
I20260808 03:54:17.112687 13826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:17.120044 13826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:17.123473 13826 hybrid_clock.cc:648] HybridClock initialized: now 1786161257123394 us; error 67 us; skew 500 ppm
I20260808 03:54:17.123952 13826 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:17 dist-test-slave-jlzn krb5kdc[12028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161257, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:17.128335 13826 init.cc:377] Logged in from keytab as kudu/127.26.146.130@KRBTEST.COM (short username kudu)
I20260808 03:54:17.130235 13826 webserver.cc:460] Webserver started at http://127.26.146.130:40615/ using document root <none> and password file <none>
I20260808 03:54:17.131181 13826 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:17.131348 13826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 03:54:17 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161257, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for HTTP/127.26.146.148@KRBTEST.COM
I20260808 03:54:17.158828 13826 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.000s sys 0.004s
I20260808 03:54:17.162551 13863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:17.164276 13826 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 03:54:17.164455 13826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "8d459f0b78c9482a8adc2f03989bd42c"
format_stamp: "Formatted at 2026-08-08 03:54:14 on dist-test-slave-jlzn"
server_key: "f0da3107a0fa0f887a5e997cbc2e2c74"
server_key_iv: "2a9265ae134313a14494e14649808573"
server_key_version: "kuduclusterkey@0"
I20260808 03:54:17.165835 13826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:17.199592 13826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:17.205435 13826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:17.205744 13826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:17.206982 13826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:54:17.208772 13826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:54:17.208880 13826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:17.208986 13826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:54:17.209070 13826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 03:54:17 dist-test-slave-jlzn krb5kdc[12028](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786161257, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:17.272058 13826 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:43063
I20260808 03:54:17.272115 14033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:43063 every 8 connection(s)
I20260808 03:54:17.273833 13826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:54:17.274705 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 13826
I20260808 03:54:17.275313 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.146.148:51089/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:45327
--local_ip_for_outbound_sockets=127.26.146.131
--tserver_master_addrs=127.26.146.190:39185
--webserver_port=43955
--webserver_interface=127.26.146.131
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.146.148:51089/kms with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 03:54:17 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161257, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:54:17.317361 14036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:17.278135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:46755 (local address 127.26.146.190:39185)
0808 03:54:17.295090 (+ 16955us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:17.295096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:17.295125 (+ 29us) server_negotiation.cc:407] Connection header received
0808 03:54:17.297823 (+ 2698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:17.297829 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:17.297926 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:17.298067 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:17.300948 (+ 2881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:17.301705 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:17.303956 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:17.304638 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:17.304779 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:17.308985 (+ 4206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:17.308993 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:17.308998 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:17.309042 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:17.312613 (+ 3571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:17.313575 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:17.313581 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:17.313585 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:17.313746 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:17.314341 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:17.314348 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:17.314351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:17.314742 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:17.315708 (+ 966us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:17.316739 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0808 03:54:17.316981 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16805,"thread_start_us":16984,"threads_started":1}
I20260808 03:54:17.319880 14037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:17.278692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:39185 (local address 127.26.146.130:46755)
0808 03:54:17.290985 (+ 12293us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:17.291044 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:17.297407 (+ 6363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:17.299061 (+ 1654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:17.299073 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:17.299904 (+ 831us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:17.300727 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:17.300755 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:17.301890 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:17.301895 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:17.303681 (+ 1786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:17.303711 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:17.304900 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:17.304907 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:17.305169 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:17.305941 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:17.305971 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:17.308701 (+ 2730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:17.312788 (+ 4087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:17.312797 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:17.312817 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:17.313409 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:17.313905 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:17.313910 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:17.313913 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:17.314196 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:17.315831 (+ 1635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:17.315840 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:17.316288 (+ 448us) client_negotiation.cc:769] Sending connection context
0808 03:54:17.317926 (+ 1638us) client_negotiation.cc:240] Negotiation successful
0808 03:54:17.318500 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11987,"thread_start_us":11777,"threads_started":1}
I20260808 03:54:17.322206 14034 heartbeater.cc:347] Connected to a master server at 127.26.146.190:39185
I20260808 03:54:17.322520 14034 heartbeater.cc:464] Registering TS with master...
I20260808 03:54:17.323349 14034 heartbeater.cc:511] Master 127.26.146.190:39185 requested a full tablet report, sending...
I20260808 03:54:17.325438 13532 ts_manager.cc:194] Registered new tserver with Master: 8d459f0b78c9482a8adc2f03989bd42c (127.26.146.130:43063)
I20260808 03:54:17.327574 13532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.130@KRBTEST.COM'} at 127.26.146.130:46755
I20260808 03:54:17.669348 13820 heartbeater.cc:464] Registering TS with master...
I20260808 03:54:17.669760 13820 heartbeater.cc:511] Master 127.26.146.190:39185 requested a full tablet report, sending...
I20260808 03:54:17.671119 13532 ts_manager.cc:194] Registered new tserver with Master: 4cd8edd17c354aaa9601089057103013 (127.26.146.129:43221)
I20260808 03:54:17.672648 13532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.129@KRBTEST.COM'} at 127.26.146.129:39319
Aug 08 03:54:17 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161257, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for HTTP/127.26.146.148@KRBTEST.COM
W20260808 03:54:17.706584 14038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:17.706914 14038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:17.707077 14038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:17.715623 14038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:17.715767 14038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:17.715973 14038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:54:17.728511 14038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.146.148:51089/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:45327
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=43955
--tserver_master_addrs=127.26.146.190:39185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:17.730216 14038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:17.732117 14038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:17.778668 14048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:17.787642 14049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:17.787921 14055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:17.788261 14038 server_base.cc:1034] running on GCE node
I20260808 03:54:17.789069 14038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:17.806077 14038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:17.813556 14038 hybrid_clock.cc:648] HybridClock initialized: now 1786161257813462 us; error 90 us; skew 500 ppm
I20260808 03:54:17.814028 14038 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:17 dist-test-slave-jlzn krb5kdc[12028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161257, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:17.818425 14038 init.cc:377] Logged in from keytab as kudu/127.26.146.131@KRBTEST.COM (short username kudu)
I20260808 03:54:17.830404 14038 webserver.cc:460] Webserver started at http://127.26.146.131:43955/ using document root <none> and password file <none>
I20260808 03:54:17.831514 14038 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:17.831710 14038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 03:54:17 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161257, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for HTTP/127.26.146.148@KRBTEST.COM
I20260808 03:54:17.866886 14038 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.002s sys 0.007s
I20260808 03:54:17.877456 14065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:17.879518 14038 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.008s sys 0.000s
I20260808 03:54:17.879727 14038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "b9dbadd80a5542cbb9483b1cb86fcd51"
format_stamp: "Formatted at 2026-08-08 03:54:14 on dist-test-slave-jlzn"
server_key: "8300bd66aebd5a79395c7d4daeb15d01"
server_key_iv: "2f394107d356726eb5a3a9be5bb573eb"
server_key_version: "kuduclusterkey@0"
I20260808 03:54:17.881325 14038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:17.932394 14038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:17.955295 14038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:17.955657 14038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:17.968299 14038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:54:17.976763 14038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:54:17.976904 14038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:17.977027 14038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:54:17.977111 14038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 03:54:18 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161257, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for HTTP/127.26.146.148@KRBTEST.COM
I20260808 03:54:18.334084 14034 heartbeater.cc:503] Master 127.26.146.190:39185 was elected leader, sending a full tablet report...
I20260808 03:54:18.366062 14233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:45327 every 8 connection(s)
I20260808 03:54:18.366458 14038 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:45327
I20260808 03:54:18.368575 14038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:54:18.372915 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 14038
I20260808 03:54:18.401504 14239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:18.373980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37732 (local address 127.26.146.190:39185)
0808 03:54:18.378297 (+ 4317us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:18.378304 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:18.379049 (+ 745us) server_negotiation.cc:407] Connection header received
0808 03:54:18.379137 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:18.379143 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:18.379239 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:18.379360 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:18.383917 (+ 4557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.385193 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.387147 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.388011 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.388234 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:18.394508 (+ 6274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:18.394515 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:18.394520 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:18.394579 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:18.398263 (+ 3684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:18.399139 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:18.399145 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:18.399149 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:18.399311 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:18.399833 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:18.399838 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:18.399841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:18.400220 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:18.400484 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:18.400905 (+ 421us) server_negotiation.cc:299] Negotiation successful
0808 03:54:18.401139 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4176,"thread_start_us":4056,"threads_started":1}
Aug 08 03:54:18 dist-test-slave-jlzn krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161257, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:54:18.416066 14243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:18.382941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:60039 (local address 127.26.146.190:39185)
0808 03:54:18.389370 (+ 6429us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:18.389376 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:18.389395 (+ 19us) server_negotiation.cc:407] Connection header received
0808 03:54:18.389445 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:18.389450 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:18.389612 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:18.389735 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:18.393409 (+ 3674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.394580 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.396928 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.397628 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.397765 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:18.406720 (+ 8955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:18.406728 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:18.406732 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:18.406770 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:18.410211 (+ 3441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:18.411189 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:18.411194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:18.411198 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:18.411340 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:18.411928 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:18.411933 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:18.411936 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:18.412298 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:18.412446 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:18.415440 (+ 2994us) server_negotiation.cc:299] Negotiation successful
0808 03:54:18.415671 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6295,"thread_start_us":5102,"threads_started":1}
I20260808 03:54:18.416543 14242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:18.382353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:39185 (local address 127.26.146.131:60039)
0808 03:54:18.383200 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:18.383252 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:18.387531 (+ 4279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:18.390213 (+ 2682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:18.390225 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:18.390882 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:18.392584 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:18.392639 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.394801 (+ 2162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.394806 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:18.396732 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:18.396749 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.398111 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.398119 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:18.398399 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:18.403830 (+ 5431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:18.403900 (+ 70us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:18.406464 (+ 2564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:18.410387 (+ 3923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:18.410397 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:18.410416 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:18.411019 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:18.411479 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:18.411484 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:18.411505 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:18.411769 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:18.412522 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:18.412531 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:18.412977 (+ 446us) client_negotiation.cc:769] Sending connection context
0808 03:54:18.413329 (+ 352us) client_negotiation.cc:240] Negotiation successful
0808 03:54:18.413906 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":637,"threads_started":1}
I20260808 03:54:18.418088 14234 heartbeater.cc:347] Connected to a master server at 127.26.146.190:39185
I20260808 03:54:18.418430 14234 heartbeater.cc:464] Registering TS with master...
I20260808 03:54:18.419199 14234 heartbeater.cc:511] Master 127.26.146.190:39185 requested a full tablet report, sending...
I20260808 03:54:18.420635 13532 ts_manager.cc:194] Registered new tserver with Master: b9dbadd80a5542cbb9483b1cb86fcd51 (127.26.146.131:45327)
I20260808 03:54:18.421744 13532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.131@KRBTEST.COM'} at 127.26.146.131:60039
I20260808 03:54:18.429991 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:54:18.446165 14243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:18.433234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37746 (local address 127.26.146.190:39185)
0808 03:54:18.433430 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:18.433437 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:18.433905 (+ 468us) server_negotiation.cc:407] Connection header received
0808 03:54:18.434076 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:18.434081 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:18.434158 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:18.434238 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:18.435771 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.436510 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.437923 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.438408 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.438520 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:18.440982 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:18.440987 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:18.440990 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:18.441022 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:18.443479 (+ 2457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:18.444172 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:18.444178 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:18.444181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:18.444307 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:18.444674 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:18.444678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:18.444680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:18.444955 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:18.445177 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:18.445738 (+ 561us) server_negotiation.cc:299] Negotiation successful
0808 03:54:18.445922 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 03:54:18.451951 13532 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37746:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:54:18.459749 13532 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:54:18.521859 14271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:18.489728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42322 (local address 127.26.146.129:43221)
0808 03:54:18.512301 (+ 22573us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:18.512312 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:18.512347 (+ 35us) server_negotiation.cc:407] Connection header received
0808 03:54:18.512476 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:18.512486 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:18.512793 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:18.513011 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:54:18.513966 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.515626 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.518705 (+ 3079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.520193 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.520399 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:18.520581 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:18.521389 (+ 808us) server_negotiation.cc:299] Negotiation successful
0808 03:54:18.521552 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22388,"thread_start_us":22212,"threads_started":1}
I20260808 03:54:18.521903 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:18.489494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:45327 (local address 127.0.0.1:40262)
0808 03:54:18.491128 (+ 1634us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:18.491145 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:18.491254 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:18.513784 (+ 22530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:18.513788 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:18.513814 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:54:18.514209 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:18.514222 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.516173 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.516181 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:18.518742 (+ 2561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:18.518753 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.520766 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.520772 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:18.521037 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:18.521113 (+ 76us) client_negotiation.cc:769] Sending connection context
0808 03:54:18.521240 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 03:54:18.521442 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1509,"mutex_wait_us":332,"thread_start_us":24722,"threads_started":1}
I20260808 03:54:18.522569 14270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:18.490588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:43063 (local address 127.0.0.1:45412)
0808 03:54:18.514495 (+ 23907us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:18.514511 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:18.514637 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:18.515417 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:18.515422 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:18.515445 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:54:18.515760 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:18.515771 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.517641 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.517646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:18.519780 (+ 2134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:18.519794 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.521819 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.521825 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:18.521998 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:18.522026 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 03:54:18.522104 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 03:54:18.522219 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23790,"mutex_wait_us":1,"thread_start_us":775,"threads_started":1}
I20260808 03:54:18.522620 14273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:18.490203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40262 (local address 127.26.146.131:45327)
0808 03:54:18.513158 (+ 22955us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:18.513168 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:18.513206 (+ 38us) server_negotiation.cc:407] Connection header received
0808 03:54:18.513312 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:18.513319 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:18.513559 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:18.513772 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:54:18.514452 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.516008 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.518905 (+ 2897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.520249 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.520403 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:18.520580 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:18.522124 (+ 1544us) server_negotiation.cc:299] Negotiation successful
0808 03:54:18.522278 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22752,"thread_start_us":22577,"threads_started":1}
I20260808 03:54:18.523459 14277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:18.491161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45412 (local address 127.26.146.130:43063)
0808 03:54:18.502504 (+ 11343us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:18.502514 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:18.514625 (+ 12111us) server_negotiation.cc:407] Connection header received
0808 03:54:18.514774 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:18.514781 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:18.515000 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:18.515187 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:54:18.516021 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.517477 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.519963 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.521307 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.521468 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:18.521617 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:18.522975 (+ 1358us) server_negotiation.cc:299] Negotiation successful
0808 03:54:18.523152 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11153,"thread_start_us":11351,"threads_started":1}
I20260808 03:54:18.522941 14269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:18.489466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:43221 (local address 127.0.0.1:42322)
0808 03:54:18.490230 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:18.490284 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:18.490464 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:18.513303 (+ 22839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:18.513309 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:18.513335 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:54:18.513693 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:18.513705 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.515787 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.515792 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:18.518553 (+ 2761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:18.518567 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.520777 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.520782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:18.521029 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:18.521114 (+ 85us) client_negotiation.cc:769] Sending connection context
0808 03:54:18.521240 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 03:54:18.522622 (+ 1382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":123,"thread_start_us":328,"threads_started":1}
I20260808 03:54:18.527660 14141 tablet_service.cc:1467] Processing CreateTablet for tablet 1c948e46b5c947f689602b6a125a2055 (DEFAULT_TABLE table=test-table [id=f29b664f2c294443b664157ae79467d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:54:18.527664 13724 tablet_service.cc:1467] Processing CreateTablet for tablet 1c948e46b5c947f689602b6a125a2055 (DEFAULT_TABLE table=test-table [id=f29b664f2c294443b664157ae79467d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:54:18.527665 13949 tablet_service.cc:1467] Processing CreateTablet for tablet 1c948e46b5c947f689602b6a125a2055 (DEFAULT_TABLE table=test-table [id=f29b664f2c294443b664157ae79467d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:54:18.529119 13724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c948e46b5c947f689602b6a125a2055. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:54:18.529124 14141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c948e46b5c947f689602b6a125a2055. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:54:18.529132 13949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c948e46b5c947f689602b6a125a2055. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:54:18.550375 14280 tablet_bootstrap.cc:492] T 1c948e46b5c947f689602b6a125a2055 P 8d459f0b78c9482a8adc2f03989bd42c: Bootstrap starting.
I20260808 03:54:18.550375 14279 tablet_bootstrap.cc:492] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013: Bootstrap starting.
I20260808 03:54:18.553232 14283 tablet_bootstrap.cc:492] T 1c948e46b5c947f689602b6a125a2055 P b9dbadd80a5542cbb9483b1cb86fcd51: Bootstrap starting.
I20260808 03:54:18.555006 14280 tablet_bootstrap.cc:654] T 1c948e46b5c947f689602b6a125a2055 P 8d459f0b78c9482a8adc2f03989bd42c: Neither blocks nor log segments found. Creating new log.
I20260808 03:54:18.556432 14280 log.cc:824] T 1c948e46b5c947f689602b6a125a2055 P 8d459f0b78c9482a8adc2f03989bd42c: Log is configured to *not* fsync() on all Append() calls
I20260808 03:54:18.557651 14283 tablet_bootstrap.cc:654] T 1c948e46b5c947f689602b6a125a2055 P b9dbadd80a5542cbb9483b1cb86fcd51: Neither blocks nor log segments found. Creating new log.
I20260808 03:54:18.558815 14283 log.cc:824] T 1c948e46b5c947f689602b6a125a2055 P b9dbadd80a5542cbb9483b1cb86fcd51: Log is configured to *not* fsync() on all Append() calls
I20260808 03:54:18.559911 14280 tablet_bootstrap.cc:492] T 1c948e46b5c947f689602b6a125a2055 P 8d459f0b78c9482a8adc2f03989bd42c: No bootstrap required, opened a new log
I20260808 03:54:18.560220 14280 ts_tablet_manager.cc:1397] T 1c948e46b5c947f689602b6a125a2055 P 8d459f0b78c9482a8adc2f03989bd42c: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260808 03:54:18.561256 14279 tablet_bootstrap.cc:654] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013: Neither blocks nor log segments found. Creating new log.
I20260808 03:54:18.562467 14279 log.cc:824] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013: Log is configured to *not* fsync() on all Append() calls
I20260808 03:54:18.572103 14279 tablet_bootstrap.cc:492] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013: No bootstrap required, opened a new log
I20260808 03:54:18.572118 14280 raft_consensus.cc:348] T 1c948e46b5c947f689602b6a125a2055 P 8d459f0b78c9482a8adc2f03989bd42c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9dbadd80a5542cbb9483b1cb86fcd51" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45327 } } peers { permanent_uuid: "4cd8edd17c354aaa9601089057103013" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43221 } } peers { permanent_uuid: "8d459f0b78c9482a8adc2f03989bd42c" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 43063 } }
I20260808 03:54:18.572467 14279 ts_tablet_manager.cc:1397] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013: Time spent bootstrapping tablet: real 0.022s user 0.006s sys 0.008s
I20260808 03:54:18.572712 14280 raft_consensus.cc:374] T 1c948e46b5c947f689602b6a125a2055 P 8d459f0b78c9482a8adc2f03989bd42c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:54:18.572870 14280 raft_consensus.cc:729] T 1c948e46b5c947f689602b6a125a2055 P 8d459f0b78c9482a8adc2f03989bd42c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d459f0b78c9482a8adc2f03989bd42c, State: Initialized, Role: FOLLOWER
I20260808 03:54:18.573510 14283 tablet_bootstrap.cc:492] T 1c948e46b5c947f689602b6a125a2055 P b9dbadd80a5542cbb9483b1cb86fcd51: No bootstrap required, opened a new log
I20260808 03:54:18.573819 14283 ts_tablet_manager.cc:1397] T 1c948e46b5c947f689602b6a125a2055 P b9dbadd80a5542cbb9483b1cb86fcd51: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.018s
I20260808 03:54:18.573715 14280 consensus_queue.cc:260] T 1c948e46b5c947f689602b6a125a2055 P 8d459f0b78c9482a8adc2f03989bd42c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9dbadd80a5542cbb9483b1cb86fcd51" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45327 } } peers { permanent_uuid: "4cd8edd17c354aaa9601089057103013" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43221 } } peers { permanent_uuid: "8d459f0b78c9482a8adc2f03989bd42c" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 43063 } }
I20260808 03:54:18.581878 14280 ts_tablet_manager.cc:1428] T 1c948e46b5c947f689602b6a125a2055 P 8d459f0b78c9482a8adc2f03989bd42c: Time spent starting tablet: real 0.022s user 0.009s sys 0.010s
I20260808 03:54:18.583312 14279 raft_consensus.cc:348] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9dbadd80a5542cbb9483b1cb86fcd51" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45327 } } peers { permanent_uuid: "4cd8edd17c354aaa9601089057103013" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43221 } } peers { permanent_uuid: "8d459f0b78c9482a8adc2f03989bd42c" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 43063 } }
I20260808 03:54:18.583873 14279 raft_consensus.cc:374] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:54:18.583619 14283 raft_consensus.cc:348] T 1c948e46b5c947f689602b6a125a2055 P b9dbadd80a5542cbb9483b1cb86fcd51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9dbadd80a5542cbb9483b1cb86fcd51" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45327 } } peers { permanent_uuid: "4cd8edd17c354aaa9601089057103013" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43221 } } peers { permanent_uuid: "8d459f0b78c9482a8adc2f03989bd42c" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 43063 } }
I20260808 03:54:18.584020 14283 raft_consensus.cc:374] T 1c948e46b5c947f689602b6a125a2055 P b9dbadd80a5542cbb9483b1cb86fcd51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:54:18.584052 14279 raft_consensus.cc:729] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd8edd17c354aaa9601089057103013, State: Initialized, Role: FOLLOWER
I20260808 03:54:18.584184 14283 raft_consensus.cc:729] T 1c948e46b5c947f689602b6a125a2055 P b9dbadd80a5542cbb9483b1cb86fcd51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9dbadd80a5542cbb9483b1cb86fcd51, State: Initialized, Role: FOLLOWER
I20260808 03:54:18.584844 14279 consensus_queue.cc:260] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9dbadd80a5542cbb9483b1cb86fcd51" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45327 } } peers { permanent_uuid: "4cd8edd17c354aaa9601089057103013" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43221 } } peers { permanent_uuid: "8d459f0b78c9482a8adc2f03989bd42c" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 43063 } }
I20260808 03:54:18.584969 14283 consensus_queue.cc:260] T 1c948e46b5c947f689602b6a125a2055 P b9dbadd80a5542cbb9483b1cb86fcd51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9dbadd80a5542cbb9483b1cb86fcd51" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45327 } } peers { permanent_uuid: "4cd8edd17c354aaa9601089057103013" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43221 } } peers { permanent_uuid: "8d459f0b78c9482a8adc2f03989bd42c" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 43063 } }
I20260808 03:54:18.592257 14234 heartbeater.cc:503] Master 127.26.146.190:39185 was elected leader, sending a full tablet report...
I20260808 03:54:18.592430 13820 heartbeater.cc:503] Master 127.26.146.190:39185 was elected leader, sending a full tablet report...
I20260808 03:54:18.593191 14283 ts_tablet_manager.cc:1428] T 1c948e46b5c947f689602b6a125a2055 P b9dbadd80a5542cbb9483b1cb86fcd51: Time spent starting tablet: real 0.019s user 0.009s sys 0.009s
I20260808 03:54:18.594357 14279 ts_tablet_manager.cc:1428] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013: Time spent starting tablet: real 0.022s user 0.010s sys 0.008s
W20260808 03:54:18.617532 13823 tablet.cc:2369] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:54:18.627931 14237 tablet.cc:2369] T 1c948e46b5c947f689602b6a125a2055 P b9dbadd80a5542cbb9483b1cb86fcd51: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:54:18.793356 14035 tablet.cc:2369] T 1c948e46b5c947f689602b6a125a2055 P 8d459f0b78c9482a8adc2f03989bd42c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:54:18.810503 14290 raft_consensus.cc:482] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:54:18.810832 14290 raft_consensus.cc:504] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9dbadd80a5542cbb9483b1cb86fcd51" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45327 } } peers { permanent_uuid: "4cd8edd17c354aaa9601089057103013" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43221 } } peers { permanent_uuid: "8d459f0b78c9482a8adc2f03989bd42c" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 43063 } }
I20260808 03:54:18.812620 14290 leader_election.cc:288] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b9dbadd80a5542cbb9483b1cb86fcd51 (127.26.146.131:45327), 8d459f0b78c9482a8adc2f03989bd42c (127.26.146.130:43063)
I20260808 03:54:18.819068 14273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:18.813101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:58887 (local address 127.26.146.131:45327)
0808 03:54:18.813317 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:18.813323 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:18.813996 (+ 673us) server_negotiation.cc:407] Connection header received
0808 03:54:18.814187 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:18.814191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:18.814263 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:18.814336 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:54:18.814981 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.815710 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.817470 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.818219 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.818361 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:18.818437 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:18.818709 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 03:54:18.818832 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 03:54:18.819279 14277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:18.813271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:55213 (local address 127.26.146.130:43063)
0808 03:54:18.813534 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:18.813541 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:18.814172 (+ 631us) server_negotiation.cc:407] Connection header received
0808 03:54:18.814337 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:18.814341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:18.814402 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:18.814478 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:54:18.815146 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.815926 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.817654 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.818405 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.818542 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:18.818616 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:18.818910 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 03:54:18.819044 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 03:54:18.819358 14293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:18.813191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:43063 (local address 127.26.146.129:55213)
0808 03:54:18.814061 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:18.814075 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:18.814206 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:18.814586 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:18.814590 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:18.814620 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:54:18.814984 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:18.814995 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.816058 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.816063 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:18.817500 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:18.817513 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.818632 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.818639 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:18.818762 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:18.818785 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 03:54:18.818863 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 03:54:18.818990 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"mutex_wait_us":21,"thread_start_us":562,"threads_started":1}
I20260808 03:54:18.819624 14292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:18.812909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:45327 (local address 127.26.146.129:58887)
0808 03:54:18.813883 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:18.813899 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:18.814053 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:18.814368 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:18.814372 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:18.814397 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:54:18.814832 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:18.814844 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.815864 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.815868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:18.817325 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:18.817337 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.818408 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.818414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:18.818560 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:18.818587 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 03:54:18.818669 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 03:54:18.818797 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":837,"mutex_wait_us":2,"thread_start_us":622,"threads_started":1}
I20260808 03:54:18.820415 13976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c948e46b5c947f689602b6a125a2055" candidate_uuid: "4cd8edd17c354aaa9601089057103013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d459f0b78c9482a8adc2f03989bd42c" is_pre_election: true
I20260808 03:54:18.820565 14169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c948e46b5c947f689602b6a125a2055" candidate_uuid: "4cd8edd17c354aaa9601089057103013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9dbadd80a5542cbb9483b1cb86fcd51" is_pre_election: true
I20260808 03:54:18.820932 13976 raft_consensus.cc:2454] T 1c948e46b5c947f689602b6a125a2055 P 8d459f0b78c9482a8adc2f03989bd42c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cd8edd17c354aaa9601089057103013 in term 0.
I20260808 03:54:18.821034 14169 raft_consensus.cc:2454] T 1c948e46b5c947f689602b6a125a2055 P b9dbadd80a5542cbb9483b1cb86fcd51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cd8edd17c354aaa9601089057103013 in term 0.
I20260808 03:54:18.821600 13655 leader_election.cc:302] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4cd8edd17c354aaa9601089057103013, 8d459f0b78c9482a8adc2f03989bd42c, b9dbadd80a5542cbb9483b1cb86fcd51; no voters:
I20260808 03:54:18.822041 14290 raft_consensus.cc:2781] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:54:18.822188 14290 raft_consensus.cc:482] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:54:18.822283 14290 raft_consensus.cc:3037] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:54:18.824536 14290 raft_consensus.cc:504] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9dbadd80a5542cbb9483b1cb86fcd51" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45327 } } peers { permanent_uuid: "4cd8edd17c354aaa9601089057103013" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43221 } } peers { permanent_uuid: "8d459f0b78c9482a8adc2f03989bd42c" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 43063 } }
I20260808 03:54:18.825085 14290 leader_election.cc:288] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013 [CANDIDATE]: Term 1 election: Requested vote from peers b9dbadd80a5542cbb9483b1cb86fcd51 (127.26.146.131:45327), 8d459f0b78c9482a8adc2f03989bd42c (127.26.146.130:43063)
I20260808 03:54:18.825506 14169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c948e46b5c947f689602b6a125a2055" candidate_uuid: "4cd8edd17c354aaa9601089057103013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9dbadd80a5542cbb9483b1cb86fcd51"
I20260808 03:54:18.825560 13976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c948e46b5c947f689602b6a125a2055" candidate_uuid: "4cd8edd17c354aaa9601089057103013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d459f0b78c9482a8adc2f03989bd42c"
I20260808 03:54:18.825672 14169 raft_consensus.cc:3037] T 1c948e46b5c947f689602b6a125a2055 P b9dbadd80a5542cbb9483b1cb86fcd51 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:54:18.825742 13976 raft_consensus.cc:3037] T 1c948e46b5c947f689602b6a125a2055 P 8d459f0b78c9482a8adc2f03989bd42c [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:54:18.827744 14169 raft_consensus.cc:2454] T 1c948e46b5c947f689602b6a125a2055 P b9dbadd80a5542cbb9483b1cb86fcd51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd8edd17c354aaa9601089057103013 in term 1.
I20260808 03:54:18.827859 13976 raft_consensus.cc:2454] T 1c948e46b5c947f689602b6a125a2055 P 8d459f0b78c9482a8adc2f03989bd42c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd8edd17c354aaa9601089057103013 in term 1.
I20260808 03:54:18.828153 13653 leader_election.cc:302] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd8edd17c354aaa9601089057103013, b9dbadd80a5542cbb9483b1cb86fcd51; no voters:
I20260808 03:54:18.828475 14290 raft_consensus.cc:2781] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:54:18.828912 14290 raft_consensus.cc:686] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013 [term 1 LEADER]: Becoming Leader. State: Replica: 4cd8edd17c354aaa9601089057103013, State: Running, Role: LEADER
I20260808 03:54:18.829404 14290 consensus_queue.cc:237] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9dbadd80a5542cbb9483b1cb86fcd51" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45327 } } peers { permanent_uuid: "4cd8edd17c354aaa9601089057103013" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43221 } } peers { permanent_uuid: "8d459f0b78c9482a8adc2f03989bd42c" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 43063 } }
I20260808 03:54:18.833485 13532 catalog_manager.cc:5495] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cd8edd17c354aaa9601089057103013 (127.26.146.129). New cstate: current_term: 1 leader_uuid: "4cd8edd17c354aaa9601089057103013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9dbadd80a5542cbb9483b1cb86fcd51" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd8edd17c354aaa9601089057103013" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d459f0b78c9482a8adc2f03989bd42c" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 43063 } health_report { overall_health: UNKNOWN } } }
I20260808 03:54:18.862815 14271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:18.857499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42334 (local address 127.26.146.129:43221)
0808 03:54:18.857688 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:18.857694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:18.857776 (+ 82us) server_negotiation.cc:407] Connection header received
0808 03:54:18.857993 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:18.857997 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:18.858075 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:18.858209 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:54:18.858604 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.859340 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.860742 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.861157 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.861265 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:18.861751 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:54:18.862252 (+ 501us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:54:18.862396 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:18.862486 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 03:54:18.862567 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 03:54:18.872668 13976 raft_consensus.cc:1261] T 1c948e46b5c947f689602b6a125a2055 P 8d459f0b78c9482a8adc2f03989bd42c [term 1 FOLLOWER]: Refusing update from remote peer 4cd8edd17c354aaa9601089057103013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:54:18.872668 14169 raft_consensus.cc:1261] T 1c948e46b5c947f689602b6a125a2055 P b9dbadd80a5542cbb9483b1cb86fcd51 [term 1 FOLLOWER]: Refusing update from remote peer 4cd8edd17c354aaa9601089057103013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:54:18.873605 14294 consensus_queue.cc:1035] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9dbadd80a5542cbb9483b1cb86fcd51" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 45327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:54:18.873925 14290 consensus_queue.cc:1035] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d459f0b78c9482a8adc2f03989bd42c" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 43063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:54:18.884141 14297 mvcc.cc:205] Tried to move back new op lower bound from 7316116516331331584 to 7316116516169023488. Current Snapshot: MvccSnapshot[applied={T|T < 7316116516331331584}]
I20260808 03:54:18.886035 14300 mvcc.cc:205] Tried to move back new op lower bound from 7316116516331331584 to 7316116516169023488. Current Snapshot: MvccSnapshot[applied={T|T < 7316116516331331584}]
I20260808 03:54:18.898940 13532 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:54:18.899178 13532 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:54:18.902874 13532 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c701d9c7d4645a09226eb2316dc1768: Sending DeleteTablet for 3 replicas of tablet 1c948e46b5c947f689602b6a125a2055
I20260808 03:54:18.904088 14141 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c948e46b5c947f689602b6a125a2055 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:54:18 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:40262
I20260808 03:54:18.904497 13724 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c948e46b5c947f689602b6a125a2055 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:54:18 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:42322
I20260808 03:54:18.904778 13949 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c948e46b5c947f689602b6a125a2055 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:54:18 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:45412
I20260808 03:54:18.910073 14315 tablet_replica.cc:330] T 1c948e46b5c947f689602b6a125a2055 P 8d459f0b78c9482a8adc2f03989bd42c: stopping tablet replica
I20260808 03:54:18.910596 14315 raft_consensus.cc:2229] T 1c948e46b5c947f689602b6a125a2055 P 8d459f0b78c9482a8adc2f03989bd42c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:54:18.911181 14315 raft_consensus.cc:2258] T 1c948e46b5c947f689602b6a125a2055 P 8d459f0b78c9482a8adc2f03989bd42c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:54:18.911978 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 13584
I20260808 03:54:18.916406 14311 tablet_replica.cc:330] T 1c948e46b5c947f689602b6a125a2055 P b9dbadd80a5542cbb9483b1cb86fcd51: stopping tablet replica
I20260808 03:54:18.916872 14311 raft_consensus.cc:2229] T 1c948e46b5c947f689602b6a125a2055 P b9dbadd80a5542cbb9483b1cb86fcd51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:54:18.917398 14311 raft_consensus.cc:2258] T 1c948e46b5c947f689602b6a125a2055 P b9dbadd80a5542cbb9483b1cb86fcd51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:54:18.919260 14315 ts_tablet_manager.cc:1905] T 1c948e46b5c947f689602b6a125a2055 P 8d459f0b78c9482a8adc2f03989bd42c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:54:18.920912 14311 ts_tablet_manager.cc:1905] T 1c948e46b5c947f689602b6a125a2055 P b9dbadd80a5542cbb9483b1cb86fcd51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:54:18.924018 14313 tablet_replica.cc:330] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013: stopping tablet replica
I20260808 03:54:18.926101 14315 ts_tablet_manager.cc:1918] T 1c948e46b5c947f689602b6a125a2055 P 8d459f0b78c9482a8adc2f03989bd42c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:54:18.926302 14315 log.cc:1196] T 1c948e46b5c947f689602b6a125a2055 P 8d459f0b78c9482a8adc2f03989bd42c: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/1c948e46b5c947f689602b6a125a2055
I20260808 03:54:18.926642 14313 raft_consensus.cc:2229] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013 [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:54:18.926959 14315 ts_tablet_manager.cc:1939] T 1c948e46b5c947f689602b6a125a2055 P 8d459f0b78c9482a8adc2f03989bd42c: Deleting consensus metadata
I20260808 03:54:18.927367 14313 raft_consensus.cc:2258] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:54:18.928150 14311 ts_tablet_manager.cc:1918] T 1c948e46b5c947f689602b6a125a2055 P b9dbadd80a5542cbb9483b1cb86fcd51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:54:18.928373 14311 log.cc:1196] T 1c948e46b5c947f689602b6a125a2055 P b9dbadd80a5542cbb9483b1cb86fcd51: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/1c948e46b5c947f689602b6a125a2055
I20260808 03:54:18.928839 13515 catalog_manager.cc:4841] TS 8d459f0b78c9482a8adc2f03989bd42c (127.26.146.130:43063): tablet 1c948e46b5c947f689602b6a125a2055 (table test-table [id=f29b664f2c294443b664157ae79467d3]) successfully deleted
I20260808 03:54:18.928929 14311 ts_tablet_manager.cc:1939] T 1c948e46b5c947f689602b6a125a2055 P b9dbadd80a5542cbb9483b1cb86fcd51: Deleting consensus metadata
I20260808 03:54:18.930626 13513 catalog_manager.cc:4841] TS b9dbadd80a5542cbb9483b1cb86fcd51 (127.26.146.131:45327): tablet 1c948e46b5c947f689602b6a125a2055 (table test-table [id=f29b664f2c294443b664157ae79467d3]) successfully deleted
I20260808 03:54:18.932065 14313 ts_tablet_manager.cc:1905] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:54:18.938495 14313 ts_tablet_manager.cc:1918] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:54:18.938669 14313 log.cc:1196] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786161058098901-27210-0/minicluster-data/ts-0/wal/wals/1c948e46b5c947f689602b6a125a2055
I20260808 03:54:18.939132 14313 ts_tablet_manager.cc:1939] T 1c948e46b5c947f689602b6a125a2055 P 4cd8edd17c354aaa9601089057103013: Deleting consensus metadata
I20260808 03:54:18.940672 13515 catalog_manager.cc:4841] TS 4cd8edd17c354aaa9601089057103013 (127.26.146.129:43221): tablet 1c948e46b5c947f689602b6a125a2055 (table test-table [id=f29b664f2c294443b664157ae79467d3]) successfully deleted
I20260808 03:54:18.942813 14271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:18.913539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42350 (local address 127.26.146.129:43221)
0808 03:54:18.913732 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:18.913739 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:18.919760 (+ 6021us) server_negotiation.cc:407] Connection header received
0808 03:54:18.920654 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:18.920661 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:18.920748 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:18.920825 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:18.927796 (+ 6971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.928939 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.931084 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:18.931786 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:18.932006 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:18.935207 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:18.935223 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:18.935242 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:18.935293 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:18.939649 (+ 4356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:18.940539 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:18.940558 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:18.940561 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:18.940746 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:18.941207 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:18.941213 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:18.941217 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:18.941665 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:18.941967 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:18.942207 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 03:54:18.942446 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 03:54:18.943493 13811 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:19.038223 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 13584
I20260808 03:54:19.058856 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 13826
I20260808 03:54:19.072331 14277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:19.059646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45422 (local address 127.26.146.130:43063)
0808 03:54:19.059852 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:19.059858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:19.059878 (+ 20us) server_negotiation.cc:407] Connection header received
0808 03:54:19.060049 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:19.060053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:19.060126 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:19.060227 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:19.061454 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:19.062165 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:19.063482 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:19.063944 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:19.064055 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:19.066492 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:19.066506 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:19.066521 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:19.066556 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:19.069341 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:19.069995 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:19.070002 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:19.070005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:19.070144 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:19.070525 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:19.070529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:19.070531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:19.070999 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:19.071254 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:19.071616 (+ 362us) server_negotiation.cc:299] Negotiation successful
0808 03:54:19.071930 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 03:54:19.072969 14029 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:19.160094 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 13826
I20260808 03:54:19.185364 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 14038
I20260808 03:54:19.198851 14273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:19.186136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40266 (local address 127.26.146.131:45327)
0808 03:54:19.186348 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:19.186353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:19.186374 (+ 21us) server_negotiation.cc:407] Connection header received
0808 03:54:19.186514 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:19.186519 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:19.186586 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:19.186677 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:19.188152 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:19.189196 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:19.190405 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:19.190787 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:19.190878 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:19.193429 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:19.193444 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:19.193461 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:19.193494 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:19.196244 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:19.196823 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:19.196830 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:19.196833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:19.196972 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:19.197285 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:19.197289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:19.197317 (+ 28us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:19.197858 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:19.198098 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:19.198284 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 03:54:19.198447 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 03:54:19.199472 14228 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:19.287384 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 14038
I20260808 03:54:19.307021 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 13489
I20260808 03:54:19.308341 13573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:19.419417 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 13489
I20260808 03:54:19.450591 27210 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 03:54:19.795078 27210 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 03:54:19.795306 27210 mini_ranger.cc:66] Stopping Ranger...
I20260808 03:54:20.139067 27210 mini_ranger.cc:68] Stopped Ranger
2026-08-08 03:54:20.139 UTC [12074] LOG: received smart shutdown request
2026-08-08 03:54:20.142 UTC [12074] LOG: background worker "logical replication launcher" (PID 12084) exited with exit code 1
2026-08-08 03:54:20.142 UTC [12078] LOG: shutting down
2026-08-08 03:54:20.279 UTC [12074] LOG: database system is shut down
2026-08-08T03:54:20Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52156 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:54:20 dist-test-slave-jlzn krb5kdc[14337](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:54:20 dist-test-slave-jlzn krb5kdc[14337](info): set up 2 sockets
Aug 08 03:54:20 dist-test-slave-jlzn krb5kdc[14337](info): commencing operation
krb5kdc: starting...
W20260808 03:54:21.571518 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:54:21 dist-test-slave-jlzn krb5kdc[14337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:54:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:54:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:54:21.832532 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:44247
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:40577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:44247,127.26.146.189:33539,127.26.146.188:41083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:54:22.163507 14360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:22.163836 14360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:22.163986 14360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:22.171159 14360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:54:22.171267 14360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:22.171352 14360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:22.171423 14360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:54:22.171502 14360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:54:22.181478 14360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:40577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:44247,127.26.146.189:33539,127.26.146.188:41083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:44247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:22.182834 14360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:22.184161 14360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:22.195621 14370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:22.195693 14366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:22.195716 14367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:22.196265 14360 server_base.cc:1034] running on GCE node
I20260808 03:54:22.197104 14360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:22.198460 14360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:22.199710 14360 hybrid_clock.cc:648] HybridClock initialized: now 1786161262199638 us; error 54 us; skew 500 ppm
I20260808 03:54:22.200073 14360 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:22 dist-test-slave-jlzn krb5kdc[14337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161262, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:22.203014 14360 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:54:22.204468 14360 webserver.cc:460] Webserver started at http://127.26.146.190:33255/ using document root <none> and password file <none>
I20260808 03:54:22.205122 14360 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:22.205235 14360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:54:22.205519 14360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:54:22.208566 14360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "a5809823663a4cc5832def7f0d4698f2"
format_stamp: "Formatted at 2026-08-08 03:54:22 on dist-test-slave-jlzn"
server_key: "1192feca136acc1da960cfd7e80c14b1"
server_key_iv: "880e9d45e355365931267cd5d3d69f39"
server_key_version: "encryptionkey@0"
I20260808 03:54:22.209338 14360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "a5809823663a4cc5832def7f0d4698f2"
format_stamp: "Formatted at 2026-08-08 03:54:22 on dist-test-slave-jlzn"
server_key: "1192feca136acc1da960cfd7e80c14b1"
server_key_iv: "880e9d45e355365931267cd5d3d69f39"
server_key_version: "encryptionkey@0"
I20260808 03:54:22.214212 14360 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:54:22.217456 14377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:22.218685 14360 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 03:54:22.218839 14360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "a5809823663a4cc5832def7f0d4698f2"
format_stamp: "Formatted at 2026-08-08 03:54:22 on dist-test-slave-jlzn"
server_key: "1192feca136acc1da960cfd7e80c14b1"
server_key_iv: "880e9d45e355365931267cd5d3d69f39"
server_key_version: "encryptionkey@0"
I20260808 03:54:22.219014 14360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:22.232039 14360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:22.236141 14360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:22.236405 14360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:22.259821 14360 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:44247
I20260808 03:54:22.259856 14455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:44247 every 8 connection(s)
I20260808 03:54:22.261225 14360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:54:22.264981 14456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:54:22.270946 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 14360
I20260808 03:54:22.271200 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:54:22.272035 27210 external_mini_cluster.cc:1428] Setting key 3bb8d4e03940e637834ae5fdc2263e9b
I20260808 03:54:22.271654 14456 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44247 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:54:22 dist-test-slave-jlzn krb5kdc[14337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161262, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:54:22.294520 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.275323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:44247 (local address 127.0.0.1:57560)
0808 03:54:22.276575 (+ 1252us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:22.276616 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:22.278327 (+ 1711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:22.279168 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:22.279173 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:22.279716 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:22.280165 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:22.280184 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:22.282116 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:22.282120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:22.283158 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:22.283170 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:22.284128 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:22.284136 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:22.284300 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:22.284902 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:22.284929 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:22.287303 (+ 2374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:22.290936 (+ 3633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:22.290946 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:22.290954 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:22.291332 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:22.291763 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:22.291766 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:22.291768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:22.291946 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:22.292788 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:22.292795 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:22.292975 (+ 180us) client_negotiation.cc:769] Sending connection context
0808 03:54:22.293167 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 03:54:22.293523 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":987,"spinlock_wait_cycles":71936,"thread_start_us":744,"threads_started":1}
I20260808 03:54:22.294548 14458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.275323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57560 (local address 127.26.146.190:44247)
0808 03:54:22.276198 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:22.276219 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:22.276754 (+ 535us) server_negotiation.cc:407] Connection header received
0808 03:54:22.278677 (+ 1923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:22.278684 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:22.278949 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:22.279153 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:22.280359 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:22.281967 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:22.283292 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:22.283890 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:22.284046 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:22.287538 (+ 3492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:22.287548 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:22.287563 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:22.287597 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:22.290821 (+ 3224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:22.291458 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:22.291467 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:22.291470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:22.291662 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:22.292067 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:22.292072 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:22.292075 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:22.292667 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:22.292873 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:22.293342 (+ 469us) server_negotiation.cc:299] Negotiation successful
0808 03:54:22.293727 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":638,"thread_start_us":395,"threads_started":1}
I20260808 03:54:22.297514 14456 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } has no permanent_uuid. Determining permanent_uuid...
I20260808 03:54:22.298588 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.298145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33539 (local address 127.0.0.1:58954)
0808 03:54:22.298327 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:22.298456 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 03:54:22.298897 14378 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33539: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.26.146.189@KRBTEST.COM; defaulting to no policy
I20260808 03:54:22.300401 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.300059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33539 (local address 127.0.0.1:58968)
0808 03:54:22.300217 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:22.300284 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 03:54:22.300601 14456 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.189:33539: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33539: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.189.keytab.
Entry for principal kudu/127.26.146.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.189.keytab.
I20260808 03:54:22.331421 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.189:33539
--webserver_interface=127.26.146.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.189.keytab
--principal=kudu/127.26.146.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:40577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:44247,127.26.146.189:33539,127.26.146.188:41083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:54:22.349915 14456 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } attempt: 1
I20260808 03:54:22.351191 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.350585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33539 (local address 127.0.0.1:58984)
0808 03:54:22.350834 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:22.350984 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 03:54:22.352301 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.351881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33539 (local address 127.0.0.1:58998)
0808 03:54:22.352033 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:22.352135 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 03:54:22.352496 14456 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.189:33539: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33539: connect: Connection refused (error 111)
I20260808 03:54:22.420747 14456 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } attempt: 2
I20260808 03:54:22.422040 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.421409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33539 (local address 127.0.0.1:59008)
0808 03:54:22.421701 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:22.421847 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 03:54:22.423164 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.422730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33539 (local address 127.0.0.1:59012)
0808 03:54:22.422891 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:22.422996 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 03:54:22.423383 14456 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.189:33539: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33539: connect: Connection refused (error 111)
I20260808 03:54:22.515174 14456 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } attempt: 3
I20260808 03:54:22.516511 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.515976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33539 (local address 127.0.0.1:59014)
0808 03:54:22.516214 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:22.516333 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 03:54:22.517822 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.517340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33539 (local address 127.0.0.1:59024)
0808 03:54:22.517518 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:22.517634 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 03:54:22.518075 14456 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.189:33539: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33539: connect: Connection refused (error 111)
I20260808 03:54:22.661312 14456 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } attempt: 4
I20260808 03:54:22.662526 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.661987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33539 (local address 127.0.0.1:59028)
0808 03:54:22.662299 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:22.662390 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260808 03:54:22.663475 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.663148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33539 (local address 127.0.0.1:59038)
0808 03:54:22.663271 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:22.663359 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 03:54:22.663663 14456 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.189:33539: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33539: connect: Connection refused (error 111)
W20260808 03:54:22.664274 14467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:22.664566 14467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:22.664695 14467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:22.671777 14467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:54:22.671912 14467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:22.671981 14467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:22.672058 14467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:54:22.672124 14467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:54:22.681952 14467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:40577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:44247,127.26.146.189:33539,127.26.146.188:41083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.189.keytab
--principal=kudu/127.26.146.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.189:33539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.146.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:22.683223 14467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:22.684614 14467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:22.696033 14473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:22.696116 14474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:22.696092 14477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:22.697209 14467 server_base.cc:1034] running on GCE node
I20260808 03:54:22.698001 14467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:22.699313 14467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:22.700531 14467 hybrid_clock.cc:648] HybridClock initialized: now 1786161262700486 us; error 26 us; skew 500 ppm
I20260808 03:54:22.700898 14467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:22 dist-test-slave-jlzn krb5kdc[14337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161262, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:22.704144 14467 init.cc:377] Logged in from keytab as kudu/127.26.146.189@KRBTEST.COM (short username kudu)
I20260808 03:54:22.705610 14467 webserver.cc:460] Webserver started at http://127.26.146.189:33899/ using document root <none> and password file <none>
I20260808 03:54:22.706413 14467 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:22.706545 14467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:54:22.706848 14467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:54:22.709858 14467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-1/data/instance:
uuid: "863769ae98e240eab0f493d61e0d36c3"
format_stamp: "Formatted at 2026-08-08 03:54:22 on dist-test-slave-jlzn"
server_key: "660e92f97728ec8b64a99d1910f4c854"
server_key_iv: "6e61a0f5e1033e0494bcfe668d4eb044"
server_key_version: "encryptionkey@0"
I20260808 03:54:22.710659 14467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-1/wal/instance:
uuid: "863769ae98e240eab0f493d61e0d36c3"
format_stamp: "Formatted at 2026-08-08 03:54:22 on dist-test-slave-jlzn"
server_key: "660e92f97728ec8b64a99d1910f4c854"
server_key_iv: "6e61a0f5e1033e0494bcfe668d4eb044"
server_key_version: "encryptionkey@0"
I20260808 03:54:22.715888 14467 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260808 03:54:22.719310 14488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:22.720578 14467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:54:22.720736 14467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-1/wal
uuid: "863769ae98e240eab0f493d61e0d36c3"
format_stamp: "Formatted at 2026-08-08 03:54:22 on dist-test-slave-jlzn"
server_key: "660e92f97728ec8b64a99d1910f4c854"
server_key_iv: "6e61a0f5e1033e0494bcfe668d4eb044"
server_key_version: "encryptionkey@0"
I20260808 03:54:22.720914 14467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:22.736797 14467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:22.740823 14467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:22.741192 14467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:22.758654 14467 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.189:33539
I20260808 03:54:22.758675 14566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.189:33539 every 8 connection(s)
I20260808 03:54:22.760007 14467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-1/data/info.pb
I20260808 03:54:22.763625 14569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:54:22.765460 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 14467
I20260808 03:54:22.765630 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-1/wal/instance
I20260808 03:54:22.766069 27210 external_mini_cluster.cc:1428] Setting key 4c24b8d35d02c6a14e83b7333adee27e
I20260808 03:54:22.769949 14569 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44247 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:54:22 dist-test-slave-jlzn krb5kdc[14337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161262, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.189@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:54:22.790840 14458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.772419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57570 (local address 127.26.146.190:44247)
0808 03:54:22.772630 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:22.772636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:22.773838 (+ 1202us) server_negotiation.cc:407] Connection header received
0808 03:54:22.776151 (+ 2313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:22.776157 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:22.776259 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:22.776370 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:22.777955 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:22.778929 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:22.780489 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:22.781076 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:22.781195 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:22.784645 (+ 3450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:22.784651 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:22.784655 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:22.784696 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:22.787836 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:22.788542 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:22.788547 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:22.788550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:22.788705 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:22.789076 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:22.789081 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:22.789084 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:22.789462 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:22.789613 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:22.790270 (+ 657us) server_negotiation.cc:299] Negotiation successful
0808 03:54:22.790498 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 03:54:22.791651 14572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.772848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:44247 (local address 127.0.0.1:57570)
0808 03:54:22.773647 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:22.773698 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:22.775871 (+ 2173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:22.776612 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:22.776620 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:22.777149 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:22.777761 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:22.777781 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:22.779072 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:22.779076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:22.780347 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:22.780360 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:22.781277 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:22.781282 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:22.781471 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:22.782123 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:22.782145 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:22.784420 (+ 2275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:22.787966 (+ 3546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:22.787973 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:22.787984 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:22.788406 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:22.788800 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:22.788803 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:22.788805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:22.788963 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:22.789568 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:22.789573 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:22.789889 (+ 316us) client_negotiation.cc:769] Sending connection context
0808 03:54:22.790105 (+ 216us) client_negotiation.cc:240] Negotiation successful
0808 03:54:22.790463 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":303,"threads_started":1}
WARNING: no policy specified for kudu/127.26.146.188@KRBTEST.COM; defaulting to no policy
I20260808 03:54:22.793299 14569 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.188@KRBTEST.COM" created.
Aug 08 03:54:22 dist-test-slave-jlzn krb5kdc[14337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161262, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.189@KRBTEST.COM for kudu/127.26.146.189@KRBTEST.COM
I20260808 03:54:22.807483 14572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.793950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33539 (local address 127.0.0.1:59048)
0808 03:54:22.794122 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:22.794140 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:22.794235 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:22.795080 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:22.795083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:22.795424 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:22.795551 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:22.795559 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:22.797322 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:22.797325 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:22.798124 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:22.798134 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:22.799132 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:22.799136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:22.799311 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:22.799836 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:22.799857 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:22.801668 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:22.805012 (+ 3344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:22.805018 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:22.805021 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:22.805484 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:22.805845 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:22.805850 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:22.805853 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:22.806011 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:22.806569 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:22.806574 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:22.806740 (+ 166us) client_negotiation.cc:769] Sending connection context
0808 03:54:22.806869 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 03:54:22.807123 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":1}
I20260808 03:54:22.807514 14579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.794075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59048 (local address 127.26.146.189:33539)
0808 03:54:22.794657 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:22.794665 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:22.794691 (+ 26us) server_negotiation.cc:407] Connection header received
0808 03:54:22.794736 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:22.794741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:22.794923 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:22.795075 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:22.795697 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:22.797201 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:22.798256 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:22.799005 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:22.799129 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:22.801881 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:22.801895 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:22.801910 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:22.801942 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:22.804883 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:22.805608 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:22.805614 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:22.805617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:22.805729 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:22.806107 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:22.806110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:22.806112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:22.806447 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:22.806618 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:22.806990 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 03:54:22.807227 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":396,"thread_start_us":250,"threads_started":1}
I20260808 03:54:22.810034 14569 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } has no permanent_uuid. Determining permanent_uuid...
I20260808 03:54:22.811297 14572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.810827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:41083 (local address 127.0.0.1:39160)
0808 03:54:22.810998 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:22.811133 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 03:54:22.811594 14493 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
I20260808 03:54:22.813172 14572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.812812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:41083 (local address 127.0.0.1:39172)
0808 03:54:22.812977 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:22.813061 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 03:54:22.813385 14569 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:41083: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.188.keytab.
Entry for principal kudu/127.26.146.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.188.keytab.
I20260808 03:54:22.826815 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.188:41083
--webserver_interface=127.26.146.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.188.keytab
--principal=kudu/127.26.146.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:40577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:44247,127.26.146.189:33539,127.26.146.188:41083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:54:22.862720 14569 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } attempt: 1
I20260808 03:54:22.864117 14572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.863494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:41083 (local address 127.0.0.1:39176)
0808 03:54:22.863772 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:22.863919 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260808 03:54:22.865819 14572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.864804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:41083 (local address 127.0.0.1:39180)
0808 03:54:22.865551 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:22.865645 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":368,"threads_started":1}
W20260808 03:54:22.866073 14569 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:41083: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
I20260808 03:54:22.934329 14569 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } attempt: 2
I20260808 03:54:22.935529 14572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.935056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:41083 (local address 127.0.0.1:39182)
0808 03:54:22.935260 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:22.935373 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 03:54:22.936713 14572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.936319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:41083 (local address 127.0.0.1:39184)
0808 03:54:22.936484 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:22.936572 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 03:54:22.936959 14569 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:41083: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
I20260808 03:54:22.962909 14456 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } attempt: 5
Aug 08 03:54:22 dist-test-slave-jlzn krb5kdc[14337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161262, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for kudu/127.26.146.189@KRBTEST.COM
I20260808 03:54:22.976172 14579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.963848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59054 (local address 127.26.146.189:33539)
0808 03:54:22.964024 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:22.964030 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:22.964050 (+ 20us) server_negotiation.cc:407] Connection header received
0808 03:54:22.964218 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:22.964222 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:22.964286 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:22.964370 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:22.965193 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:22.965883 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:22.967146 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:22.967606 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:22.967724 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:22.970994 (+ 3270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:22.970999 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:22.971002 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:22.971035 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:22.973545 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:22.974312 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:22.974317 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:22.974319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:22.974451 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:22.974877 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:22.974881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:22.974883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:22.975178 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:22.975291 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:22.975739 (+ 448us) server_negotiation.cc:299] Negotiation successful
0808 03:54:22.975945 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 03:54:22.976297 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.963657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33539 (local address 127.0.0.1:59054)
0808 03:54:22.963908 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:22.963929 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:22.964075 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:22.964420 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:22.964425 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:22.964826 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:22.965021 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:22.965030 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:22.966022 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:22.966033 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:22.966995 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:22.967009 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:22.967749 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:22.967756 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:22.967960 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:22.968540 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:22.968561 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:22.970796 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:22.973686 (+ 2890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:22.973692 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:22.973696 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:22.974179 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:22.974586 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:22.974590 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:22.974594 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:22.974771 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:22.975287 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:22.975292 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:22.975459 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 03:54:22.975611 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 03:54:22.975954 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 03:54:22.977402 14456 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } has no permanent_uuid. Determining permanent_uuid...
I20260808 03:54:22.978634 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.978247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:41083 (local address 127.0.0.1:39200)
0808 03:54:22.978411 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:22.978490 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 03:54:22.979615 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:22.979314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:41083 (local address 127.0.0.1:39204)
0808 03:54:22.979426 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:22.979517 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 03:54:22.979843 14456 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:41083: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
I20260808 03:54:23.028162 14569 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } attempt: 3
I20260808 03:54:23.029277 14572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.028825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:41083 (local address 127.0.0.1:39210)
0808 03:54:23.029007 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:23.029111 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 03:54:23.030339 14572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.029917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:41083 (local address 127.0.0.1:39214)
0808 03:54:23.030145 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:23.030217 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260808 03:54:23.030542 14569 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:41083: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
I20260808 03:54:23.031044 14456 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } attempt: 1
I20260808 03:54:23.031967 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.031559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:41083 (local address 127.0.0.1:39224)
0808 03:54:23.031709 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:23.031785 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 03:54:23.032790 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.032491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:41083 (local address 127.0.0.1:39232)
0808 03:54:23.032623 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:23.032695 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 03:54:23.032991 14456 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:41083: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
I20260808 03:54:23.101989 14456 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } attempt: 2
I20260808 03:54:23.103091 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.102592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:41083 (local address 127.0.0.1:39248)
0808 03:54:23.102823 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:23.102909 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 03:54:23.104089 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.103731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:41083 (local address 127.0.0.1:39250)
0808 03:54:23.103909 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:23.103978 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 03:54:23.104349 14456 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:41083: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
I20260808 03:54:23.173772 14569 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } attempt: 4
I20260808 03:54:23.174885 14572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.174412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:41083 (local address 127.0.0.1:39252)
0808 03:54:23.174621 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:23.174732 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 03:54:23.175977 14572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.175625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:41083 (local address 127.0.0.1:39260)
0808 03:54:23.175764 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:23.175860 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 03:54:23.176222 14569 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:41083: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
W20260808 03:54:23.197945 14584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:23.198254 14584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:23.198391 14584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:23.205592 14584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:54:23.205706 14584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:23.205785 14584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:23.205859 14584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:54:23.205919 14584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:54:23.210566 14456 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } attempt: 3
I20260808 03:54:23.211884 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.211285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:41083 (local address 127.0.0.1:39276)
0808 03:54:23.211533 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:23.211643 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 03:54:23.213120 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.212637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:41083 (local address 127.0.0.1:39290)
0808 03:54:23.212864 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:23.212957 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260808 03:54:23.213390 14456 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:41083: Network error: Client connection negotiation failed: client connection to 127.26.146.188:41083: connect: Connection refused (error 111)
I20260808 03:54:23.216516 14584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:40577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:44247,127.26.146.189:33539,127.26.146.188:41083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.188.keytab
--principal=kudu/127.26.146.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.188:41083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.146.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:23.217816 14584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:23.219265 14584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:23.231176 14594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:23.231380 14599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:23.231956 14592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:23.233614 14584 server_base.cc:1034] running on GCE node
I20260808 03:54:23.234417 14584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:23.235781 14584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:23.237051 14584 hybrid_clock.cc:648] HybridClock initialized: now 1786161263236982 us; error 47 us; skew 500 ppm
I20260808 03:54:23.237387 14584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:23 dist-test-slave-jlzn krb5kdc[14337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161263, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:23.240581 14584 init.cc:377] Logged in from keytab as kudu/127.26.146.188@KRBTEST.COM (short username kudu)
I20260808 03:54:23.242062 14584 webserver.cc:460] Webserver started at http://127.26.146.188:44973/ using document root <none> and password file <none>
I20260808 03:54:23.242715 14584 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:23.242820 14584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:54:23.243105 14584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:54:23.246106 14584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-2/data/instance:
uuid: "b6e9a6d170394d21b67b5badee1c99fe"
format_stamp: "Formatted at 2026-08-08 03:54:23 on dist-test-slave-jlzn"
server_key: "faf6b6b76aa586cd98116404441397f0"
server_key_iv: "722cab786dc8d73a33927e5bedea4943"
server_key_version: "encryptionkey@0"
I20260808 03:54:23.246882 14584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-2/wal/instance:
uuid: "b6e9a6d170394d21b67b5badee1c99fe"
format_stamp: "Formatted at 2026-08-08 03:54:23 on dist-test-slave-jlzn"
server_key: "faf6b6b76aa586cd98116404441397f0"
server_key_iv: "722cab786dc8d73a33927e5bedea4943"
server_key_version: "encryptionkey@0"
I20260808 03:54:23.252944 14584 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.002s
I20260808 03:54:23.256832 14611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:23.258241 14584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260808 03:54:23.258399 14584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-2/wal
uuid: "b6e9a6d170394d21b67b5badee1c99fe"
format_stamp: "Formatted at 2026-08-08 03:54:23 on dist-test-slave-jlzn"
server_key: "faf6b6b76aa586cd98116404441397f0"
server_key_iv: "722cab786dc8d73a33927e5bedea4943"
server_key_version: "encryptionkey@0"
I20260808 03:54:23.258631 14584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:23.284042 14584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:23.288265 14584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:23.288538 14584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:23.307281 14584 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.188:41083
I20260808 03:54:23.307351 14698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.188:41083 every 8 connection(s)
I20260808 03:54:23.308674 14584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-2/data/info.pb
I20260808 03:54:23.311574 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 14584
I20260808 03:54:23.311754 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786161058098901-27210-0/minicluster-data/master-2/wal/instance
I20260808 03:54:23.312235 27210 external_mini_cluster.cc:1428] Setting key d0dc9c9d408face7b23b4e2e6e39bdda
I20260808 03:54:23.313331 14702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:54:23.322604 14702 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44247 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:54:23 dist-test-slave-jlzn krb5kdc[14337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161263, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.188@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
Aug 08 03:54:23 dist-test-slave-jlzn krb5kdc[14337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.188@KRBTEST.COM
I20260808 03:54:23.342306 14710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.325696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57576 (local address 127.26.146.190:44247)
0808 03:54:23.326183 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:23.326189 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:23.326691 (+ 502us) server_negotiation.cc:407] Connection header received
0808 03:54:23.327251 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:23.327256 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:23.327328 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:23.327450 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:23.329082 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.330220 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.332067 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.332533 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.332665 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:23.336413 (+ 3748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:23.336417 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:23.336421 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:23.336454 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:23.339097 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:23.340087 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:23.340092 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:23.340094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:23.340221 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:23.340684 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:23.340687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:23.340689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:23.340945 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:23.341052 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:23.341744 (+ 692us) server_negotiation.cc:299] Negotiation successful
0808 03:54:23.341915 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":225,"threads_started":1}
I20260808 03:54:23.343374 14711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.325752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:44247 (local address 127.0.0.1:57576)
0808 03:54:23.326544 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:23.326592 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:23.327037 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:23.327505 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:23.327512 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:23.328335 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:23.328867 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:23.328892 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.330366 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.330370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:23.331933 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:23.331953 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.332747 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.332752 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:23.333005 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:23.333690 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:23.333727 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:23.336234 (+ 2507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:23.339229 (+ 2995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:23.339239 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:23.339248 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:23.339962 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:23.340337 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:23.340341 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:23.340345 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:23.340586 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:23.341062 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:23.341070 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:23.341398 (+ 328us) client_negotiation.cc:769] Sending connection context
0808 03:54:23.341629 (+ 231us) client_negotiation.cc:240] Negotiation successful
0808 03:54:23.342070 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"mutex_wait_us":51,"thread_start_us":376,"threads_started":1}
I20260808 03:54:23.343881 14706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.314115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39300 (local address 127.26.146.188:41083)
0808 03:54:23.324291 (+ 10176us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:23.324307 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:23.324891 (+ 584us) server_negotiation.cc:407] Connection header received
0808 03:54:23.327040 (+ 2149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:23.327061 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:23.327309 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:23.327500 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:23.329338 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.330830 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.332500 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.333213 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.333315 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:23.337683 (+ 4368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:23.337691 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:23.337703 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:23.337735 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:23.341038 (+ 3303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:23.341659 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:23.341666 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:23.341669 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:23.341817 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:23.342256 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:23.342261 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:23.342264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:23.342700 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:23.343007 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:23.343237 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 03:54:23.343466 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":9928,"thread_start_us":9701,"threads_started":1}
I20260808 03:54:23.345052 14702 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:54:23 dist-test-slave-jlzn krb5kdc[14337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161263, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.188@KRBTEST.COM for kudu/127.26.146.189@KRBTEST.COM
Aug 08 03:54:23 dist-test-slave-jlzn krb5kdc[14337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:54:23.358361 14711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.345680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33539 (local address 127.0.0.1:59058)
0808 03:54:23.345838 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:23.345857 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:23.345946 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:23.346624 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:23.346629 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:23.347059 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:23.347259 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:23.347270 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.348236 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.348241 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:23.349582 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:23.349594 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.350268 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.350273 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:23.350478 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:23.350967 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:23.350997 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:23.353135 (+ 2138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:23.356165 (+ 3030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:23.356169 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:23.356171 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:23.356498 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:23.356882 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:23.356885 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:23.356887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:23.357013 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:23.357557 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:23.357560 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:23.357671 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 03:54:23.357788 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 03:54:23.358028 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 03:54:23.358472 14579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.345766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59058 (local address 127.26.146.189:33539)
0808 03:54:23.345921 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:23.345927 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:23.345947 (+ 20us) server_negotiation.cc:407] Connection header received
0808 03:54:23.346406 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:23.346412 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:23.346494 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:23.346589 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:23.347386 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.348107 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.349708 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.350130 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.350239 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:23.353310 (+ 3071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:23.353315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:23.353318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:23.353349 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:23.356058 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:23.356613 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:23.356619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:23.356622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:23.356785 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:23.357114 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:23.357118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:23.357122 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:23.357447 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:23.357624 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:23.357921 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 03:54:23.358128 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 03:54:23.359390 14702 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } has no permanent_uuid. Determining permanent_uuid...
I20260808 03:54:23.360529 14710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.346625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57586 (local address 127.26.146.190:44247)
0808 03:54:23.346785 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:23.346790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:23.346899 (+ 109us) server_negotiation.cc:407] Connection header received
0808 03:54:23.347025 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:23.347030 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:23.347115 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:23.347220 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:23.348312 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.349332 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.350475 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.351026 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.351154 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:23.354887 (+ 3733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:23.354892 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:23.354895 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:23.354923 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:23.357689 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:23.358294 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:23.358299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:23.358303 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:23.358426 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:23.359032 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:23.359037 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:23.359039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:23.359382 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:23.359630 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:23.360019 (+ 389us) server_negotiation.cc:299] Negotiation successful
0808 03:54:23.360237 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 03:54:23 dist-test-slave-jlzn krb5kdc[14337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161263, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.188@KRBTEST.COM for kudu/127.26.146.188@KRBTEST.COM
Aug 08 03:54:23 dist-test-slave-jlzn krb5kdc[14337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.189@KRBTEST.COM
I20260808 03:54:23.373198 14711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.360346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:41083 (local address 127.0.0.1:39308)
0808 03:54:23.360509 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:23.360524 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:23.360597 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:23.360954 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:23.360959 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:23.361376 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:23.361553 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:23.361564 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.362805 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.362811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:23.364167 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:23.364181 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.365218 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.365222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:23.365352 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:23.365846 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:23.365867 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:23.368077 (+ 2210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:23.370772 (+ 2695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:23.370778 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:23.370781 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:23.371264 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:23.371752 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:23.371755 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:23.371757 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:23.371925 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:23.372482 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:23.372485 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:23.372603 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 03:54:23.372724 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 03:54:23.372926 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 03:54:23.373562 14706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.360251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39308 (local address 127.26.146.188:41083)
0808 03:54:23.360690 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:23.360703 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:23.360723 (+ 20us) server_negotiation.cc:407] Connection header received
0808 03:54:23.360768 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:23.360772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:23.360833 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:23.360930 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:23.361705 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.362697 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.364318 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.365080 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.365205 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:23.368235 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:23.368239 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:23.368243 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:23.368271 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:23.370652 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:23.371377 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:23.371381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:23.371383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:23.371625 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:23.372026 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:23.372031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:23.372034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:23.372358 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:23.372505 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:23.372872 (+ 367us) server_negotiation.cc:299] Negotiation successful
0808 03:54:23.373113 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335}
I20260808 03:54:23.377017 14579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.362190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59068 (local address 127.26.146.189:33539)
0808 03:54:23.362370 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:23.362375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:23.362394 (+ 19us) server_negotiation.cc:407] Connection header received
0808 03:54:23.362555 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:23.362560 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:23.362623 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:23.362713 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:23.364318 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.365354 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.366997 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.367385 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.367498 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:23.371536 (+ 4038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:23.371542 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:23.371547 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:23.371585 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:23.374461 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:23.375205 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:23.375210 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:23.375213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:23.375374 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:23.375753 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:23.375758 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:23.375761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:23.376098 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:23.376378 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:23.376490 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 03:54:23.376697 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 03:54:23.379101 14702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6e9a6d170394d21b67b5badee1c99fe: Bootstrap starting.
I20260808 03:54:23.382089 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:54:23.382715 14702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6e9a6d170394d21b67b5badee1c99fe: Neither blocks nor log segments found. Creating new log.
I20260808 03:54:23.383689 14702 log.cc:824] T 00000000000000000000000000000000 P b6e9a6d170394d21b67b5badee1c99fe: Log is configured to *not* fsync() on all Append() calls
I20260808 03:54:23.386598 14702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6e9a6d170394d21b67b5badee1c99fe: No bootstrap required, opened a new log
I20260808 03:54:23.390588 14456 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } attempt: 4
I20260808 03:54:23.397253 14702 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6e9a6d170394d21b67b5badee1c99fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5809823663a4cc5832def7f0d4698f2" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44247 } } peers { permanent_uuid: "863769ae98e240eab0f493d61e0d36c3" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } } peers { permanent_uuid: "b6e9a6d170394d21b67b5badee1c99fe" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } }
I20260808 03:54:23.397802 14702 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6e9a6d170394d21b67b5badee1c99fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:54:23.397965 14702 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6e9a6d170394d21b67b5badee1c99fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6e9a6d170394d21b67b5badee1c99fe, State: Initialized, Role: FOLLOWER
I20260808 03:54:23.398664 14702 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6e9a6d170394d21b67b5badee1c99fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5809823663a4cc5832def7f0d4698f2" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44247 } } peers { permanent_uuid: "863769ae98e240eab0f493d61e0d36c3" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } } peers { permanent_uuid: "b6e9a6d170394d21b67b5badee1c99fe" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } }
Aug 08 03:54:23 dist-test-slave-jlzn krb5kdc[14337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161262, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for kudu/127.26.146.188@KRBTEST.COM
I20260808 03:54:23.409446 14579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.387454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59076 (local address 127.26.146.189:33539)
0808 03:54:23.387636 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:23.387641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:23.388276 (+ 635us) server_negotiation.cc:407] Connection header received
0808 03:54:23.388333 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:23.388338 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:23.388403 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:23.388486 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:23.390811 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.394005 (+ 3194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.396106 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.396731 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.396826 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:23.402615 (+ 5789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:23.402622 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:23.402627 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:23.402674 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:23.406005 (+ 3331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:23.406989 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:23.406995 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:23.406999 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:23.407168 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:23.407624 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:23.407629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:23.407632 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:23.408026 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:23.408258 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:23.408850 (+ 592us) server_negotiation.cc:299] Negotiation successful
0808 03:54:23.409088 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 03:54:23.409636 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.391148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:41083 (local address 127.0.0.1:39322)
0808 03:54:23.391522 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:23.391542 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:23.391648 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:23.394309 (+ 2661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:23.394314 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:23.394849 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:23.395060 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:23.395073 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.396936 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.396941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:23.398264 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:23.398281 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.399321 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.399329 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:23.399520 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:23.400237 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:23.400262 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:23.402790 (+ 2528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:23.406448 (+ 3658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:23.406457 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:23.406461 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:23.407159 (+ 698us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:23.407569 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:23.407574 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:23.407577 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:23.407799 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:23.408684 (+ 885us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:23.408689 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:23.408885 (+ 196us) client_negotiation.cc:769] Sending connection context
0808 03:54:23.409013 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 03:54:23.409266 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"spinlock_wait_cycles":1280}
I20260808 03:54:23.410646 14723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.391871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39322 (local address 127.26.146.188:41083)
0808 03:54:23.393200 (+ 1329us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:23.393206 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:23.393229 (+ 23us) server_negotiation.cc:407] Connection header received
0808 03:54:23.393294 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:23.393298 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:23.393371 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:23.395239 (+ 1868us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:23.395534 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.396809 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.398430 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.399124 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.399945 (+ 821us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:23.402998 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:23.403004 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:23.403007 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:23.403042 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:23.406306 (+ 3264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:23.407264 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:23.407268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:23.407271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:23.407429 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:23.407961 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:23.407966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:23.407970 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:23.408328 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:23.409907 (+ 1579us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:23.410027 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 03:54:23.410230 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1197,"thread_start_us":933,"threads_started":1}
I20260808 03:54:23.411315 14710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.386877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57600 (local address 127.26.146.190:44247)
0808 03:54:23.387061 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:23.387068 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:23.388055 (+ 987us) server_negotiation.cc:407] Connection header received
0808 03:54:23.388351 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:23.388357 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:23.388438 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:23.388539 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:23.391108 (+ 2569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.392306 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.394257 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.394942 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.395042 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:23.404401 (+ 9359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:23.404408 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:23.404413 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:23.404454 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:23.407859 (+ 3405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:23.408803 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:23.408808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:23.408815 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:23.408973 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:23.409426 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:23.409431 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:23.409434 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:23.409970 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:23.410191 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:23.410773 (+ 582us) server_negotiation.cc:299] Negotiation successful
0808 03:54:23.410973 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 03:54:23.414088 14724 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e9a6d170394d21b67b5badee1c99fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5809823663a4cc5832def7f0d4698f2" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44247 } } peers { permanent_uuid: "863769ae98e240eab0f493d61e0d36c3" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } } peers { permanent_uuid: "b6e9a6d170394d21b67b5badee1c99fe" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } } }
I20260808 03:54:23.414575 14724 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e9a6d170394d21b67b5badee1c99fe [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:54:23.416162 14706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.388014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39312 (local address 127.26.146.188:41083)
0808 03:54:23.388184 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:23.388189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:23.398897 (+ 10708us) server_negotiation.cc:407] Connection header received
0808 03:54:23.399345 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:23.399350 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:23.399425 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:23.399490 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:23.401481 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.402463 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.404469 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.405192 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.405284 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:23.409387 (+ 4103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:23.409393 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:23.409398 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:23.409434 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:23.412787 (+ 3353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:23.413791 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:23.413796 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:23.413800 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:23.413999 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:23.414449 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:23.414454 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:23.414457 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:23.414821 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:23.415061 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:23.415577 (+ 516us) server_negotiation.cc:299] Negotiation successful
0808 03:54:23.415793 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 03:54:23.416355 14702 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6e9a6d170394d21b67b5badee1c99fe [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:54:23.416956 14456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5809823663a4cc5832def7f0d4698f2: Bootstrap starting.
I20260808 03:54:23.420583 14456 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5809823663a4cc5832def7f0d4698f2: Neither blocks nor log segments found. Creating new log.
I20260808 03:54:23.422080 14456 log.cc:824] T 00000000000000000000000000000000 P a5809823663a4cc5832def7f0d4698f2: Log is configured to *not* fsync() on all Append() calls
I20260808 03:54:23.426218 14456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5809823663a4cc5832def7f0d4698f2: No bootstrap required, opened a new log
W20260808 03:54:23.430789 14740 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b6e9a6d170394d21b67b5badee1c99fe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 03:54:23.430965 14740 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 03:54:23.426146 14584 master_runner.cc:186] Error getting master registration for 127.26.146.190:44247: OK, instance_id { permanent_uuid: "a5809823663a4cc5832def7f0d4698f2" instance_seqno: 1786161262254869 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 03:54:23.434963 14584 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 03:54:23.436101 14456 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5809823663a4cc5832def7f0d4698f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5809823663a4cc5832def7f0d4698f2" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44247 } } peers { permanent_uuid: "863769ae98e240eab0f493d61e0d36c3" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } } peers { permanent_uuid: "b6e9a6d170394d21b67b5badee1c99fe" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } }
I20260808 03:54:23.436581 14456 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5809823663a4cc5832def7f0d4698f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:54:23.436753 14456 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5809823663a4cc5832def7f0d4698f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5809823663a4cc5832def7f0d4698f2, State: Initialized, Role: FOLLOWER
I20260808 03:54:23.437497 14456 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5809823663a4cc5832def7f0d4698f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5809823663a4cc5832def7f0d4698f2" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44247 } } peers { permanent_uuid: "863769ae98e240eab0f493d61e0d36c3" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } } peers { permanent_uuid: "b6e9a6d170394d21b67b5badee1c99fe" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } }
I20260808 03:54:23.438357 14743 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5809823663a4cc5832def7f0d4698f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5809823663a4cc5832def7f0d4698f2" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44247 } } peers { permanent_uuid: "863769ae98e240eab0f493d61e0d36c3" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } } peers { permanent_uuid: "b6e9a6d170394d21b67b5badee1c99fe" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } } }
I20260808 03:54:23.438686 14743 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5809823663a4cc5832def7f0d4698f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:54:23.439591 14456 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5809823663a4cc5832def7f0d4698f2 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 03:54:23.450635 14760 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a5809823663a4cc5832def7f0d4698f2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 03:54:23.450850 14760 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 03:54:23.448506 14360 master_runner.cc:186] Error getting master registration for 127.26.146.189:33539: OK, instance_id { permanent_uuid: "863769ae98e240eab0f493d61e0d36c3" instance_seqno: 1786161262753727 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 03:54:23.460172 14360 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 03:54:23.475457 14569 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } attempt: 5
Aug 08 03:54:23 dist-test-slave-jlzn krb5kdc[14337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161262, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.189@KRBTEST.COM for kudu/127.26.146.188@KRBTEST.COM
W20260808 03:54:23.489178 14385 tablet.cc:2369] T 00000000000000000000000000000000 P a5809823663a4cc5832def7f0d4698f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:54:23.493006 14572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.476122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:41083 (local address 127.0.0.1:39334)
0808 03:54:23.476424 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:23.476444 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:23.476573 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:23.476951 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:23.476956 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:23.477511 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:23.477726 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:23.477739 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.479235 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.479241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:23.480713 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:23.480728 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.481961 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.481967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:23.482171 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:23.483293 (+ 1122us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:23.483316 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:23.486204 (+ 2888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:23.490170 (+ 3966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:23.490176 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:23.490180 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:23.490671 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:23.491072 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:23.491077 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:23.491081 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:23.491277 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:23.492009 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:23.492014 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:23.492176 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 03:54:23.492354 (+ 178us) client_negotiation.cc:240] Negotiation successful
0808 03:54:23.492630 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160}
I20260808 03:54:23.494999 14706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:23.476215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39334 (local address 127.26.146.188:41083)
0808 03:54:23.476431 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:23.476438 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:23.476504 (+ 66us) server_negotiation.cc:407] Connection header received
0808 03:54:23.476709 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:23.476715 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:23.476790 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:23.476892 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:23.477896 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.479079 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.480900 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:23.481624 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:23.481747 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:23.486398 (+ 4651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:23.486405 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:23.486410 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:23.486447 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:23.490019 (+ 3572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:23.490794 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:23.490800 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:23.490804 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:23.490961 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:23.491392 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:23.491397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:23.491401 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:23.491770 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:23.491944 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:23.492484 (+ 540us) server_negotiation.cc:299] Negotiation successful
0808 03:54:23.492727 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 03:54:23.500784 14569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3: Bootstrap starting.
I20260808 03:54:23.504941 14569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3: Neither blocks nor log segments found. Creating new log.
I20260808 03:54:23.506430 14569 log.cc:824] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3: Log is configured to *not* fsync() on all Append() calls
I20260808 03:54:23.511377 14569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3: No bootstrap required, opened a new log
I20260808 03:54:23.522212 14569 raft_consensus.cc:348] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5809823663a4cc5832def7f0d4698f2" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44247 } } peers { permanent_uuid: "863769ae98e240eab0f493d61e0d36c3" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } } peers { permanent_uuid: "b6e9a6d170394d21b67b5badee1c99fe" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } }
I20260808 03:54:23.522706 14569 raft_consensus.cc:374] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:54:23.522872 14569 raft_consensus.cc:729] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 863769ae98e240eab0f493d61e0d36c3, State: Initialized, Role: FOLLOWER
I20260808 03:54:23.523623 14569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5809823663a4cc5832def7f0d4698f2" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44247 } } peers { permanent_uuid: "863769ae98e240eab0f493d61e0d36c3" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } } peers { permanent_uuid: "b6e9a6d170394d21b67b5badee1c99fe" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } }
I20260808 03:54:23.525213 14764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5809823663a4cc5832def7f0d4698f2" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44247 } } peers { permanent_uuid: "863769ae98e240eab0f493d61e0d36c3" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } } peers { permanent_uuid: "b6e9a6d170394d21b67b5badee1c99fe" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } } }
I20260808 03:54:23.525694 14764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:54:23.526450 14569 sys_catalog.cc:564] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 03:54:23.540606 14621 tablet.cc:2369] T 00000000000000000000000000000000 P b6e9a6d170394d21b67b5badee1c99fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:54:23.542543 14467 master_runner.cc:263] No leader master found from master 863769ae98e240eab0f493d61e0d36c3
I20260808 03:54:23.542690 14467 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 03:54:23.545349 14791 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 03:54:23.545516 14791 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 03:54:23.567960 14764 raft_consensus.cc:482] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:54:23.568291 14764 raft_consensus.cc:504] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5809823663a4cc5832def7f0d4698f2" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44247 } } peers { permanent_uuid: "863769ae98e240eab0f493d61e0d36c3" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } } peers { permanent_uuid: "b6e9a6d170394d21b67b5badee1c99fe" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } }
I20260808 03:54:23.569986 14764 leader_election.cc:288] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5809823663a4cc5832def7f0d4698f2 (127.26.146.190:44247), b6e9a6d170394d21b67b5badee1c99fe (127.26.146.188:41083)
I20260808 03:54:23.572592 14658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "863769ae98e240eab0f493d61e0d36c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6e9a6d170394d21b67b5badee1c99fe" is_pre_election: true
I20260808 03:54:23.573185 14658 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b6e9a6d170394d21b67b5badee1c99fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 863769ae98e240eab0f493d61e0d36c3 in term 0.
I20260808 03:54:23.576313 14493 leader_election.cc:302] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 863769ae98e240eab0f493d61e0d36c3, b6e9a6d170394d21b67b5badee1c99fe; no voters:
I20260808 03:54:23.571367 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "863769ae98e240eab0f493d61e0d36c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5809823663a4cc5832def7f0d4698f2" is_pre_election: true
I20260808 03:54:23.579216 14419 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a5809823663a4cc5832def7f0d4698f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 863769ae98e240eab0f493d61e0d36c3 in term 0.
I20260808 03:54:23.580261 14764 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:54:23.580417 14764 raft_consensus.cc:482] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:54:23.580583 14764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:54:23.584694 14764 raft_consensus.cc:504] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5809823663a4cc5832def7f0d4698f2" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44247 } } peers { permanent_uuid: "863769ae98e240eab0f493d61e0d36c3" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } } peers { permanent_uuid: "b6e9a6d170394d21b67b5badee1c99fe" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } }
I20260808 03:54:23.585939 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "863769ae98e240eab0f493d61e0d36c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5809823663a4cc5832def7f0d4698f2"
I20260808 03:54:23.586160 14419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5809823663a4cc5832def7f0d4698f2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:54:23.589622 14419 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a5809823663a4cc5832def7f0d4698f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 863769ae98e240eab0f493d61e0d36c3 in term 1.
I20260808 03:54:23.590111 14492 leader_election.cc:302] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 863769ae98e240eab0f493d61e0d36c3, a5809823663a4cc5832def7f0d4698f2; no voters:
I20260808 03:54:23.597834 14658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "863769ae98e240eab0f493d61e0d36c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6e9a6d170394d21b67b5badee1c99fe"
I20260808 03:54:23.598059 14658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6e9a6d170394d21b67b5badee1c99fe [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:54:23.601104 14793 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:54:23.601562 14658 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b6e9a6d170394d21b67b5badee1c99fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 863769ae98e240eab0f493d61e0d36c3 in term 1.
I20260808 03:54:23.602190 14764 leader_election.cc:288] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [CANDIDATE]: Term 1 election: Requested vote from peers a5809823663a4cc5832def7f0d4698f2 (127.26.146.190:44247), b6e9a6d170394d21b67b5badee1c99fe (127.26.146.188:41083)
I20260808 03:54:23.607045 14793 raft_consensus.cc:686] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [term 1 LEADER]: Becoming Leader. State: Replica: 863769ae98e240eab0f493d61e0d36c3, State: Running, Role: LEADER
I20260808 03:54:23.607751 14793 consensus_queue.cc:237] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5809823663a4cc5832def7f0d4698f2" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44247 } } peers { permanent_uuid: "863769ae98e240eab0f493d61e0d36c3" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } } peers { permanent_uuid: "b6e9a6d170394d21b67b5badee1c99fe" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } }
I20260808 03:54:23.618278 14764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 863769ae98e240eab0f493d61e0d36c3. Latest consensus state: current_term: 1 leader_uuid: "863769ae98e240eab0f493d61e0d36c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5809823663a4cc5832def7f0d4698f2" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44247 } } peers { permanent_uuid: "863769ae98e240eab0f493d61e0d36c3" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } } peers { permanent_uuid: "b6e9a6d170394d21b67b5badee1c99fe" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } } }
I20260808 03:54:23.618597 14764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [sys.catalog]: This master's current role is: LEADER
I20260808 03:54:23.624315 14796 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:54:23.626564 14796 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:54:23.652207 14419 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a5809823663a4cc5832def7f0d4698f2 [term 1 FOLLOWER]: Refusing update from remote peer 863769ae98e240eab0f493d61e0d36c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:54:23.654927 14764 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5809823663a4cc5832def7f0d4698f2" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260808 03:54:23.655203 14658 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b6e9a6d170394d21b67b5badee1c99fe [term 1 FOLLOWER]: Refusing update from remote peer 863769ae98e240eab0f493d61e0d36c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:54:23.660501 14764 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6e9a6d170394d21b67b5badee1c99fe" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:54:23.690439 14724 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e9a6d170394d21b67b5badee1c99fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 863769ae98e240eab0f493d61e0d36c3. Latest consensus state: current_term: 1 leader_uuid: "863769ae98e240eab0f493d61e0d36c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5809823663a4cc5832def7f0d4698f2" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44247 } } peers { permanent_uuid: "863769ae98e240eab0f493d61e0d36c3" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } } peers { permanent_uuid: "b6e9a6d170394d21b67b5badee1c99fe" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } } }
I20260808 03:54:23.690778 14724 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e9a6d170394d21b67b5badee1c99fe [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:54:23.695245 14797 mvcc.cc:205] Tried to move back new op lower bound from 7316116535908233216 to 7316116535743315968. Current Snapshot: MvccSnapshot[applied={T|T < 7316116535908233216}]
I20260808 03:54:23.710325 14724 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e9a6d170394d21b67b5badee1c99fe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "863769ae98e240eab0f493d61e0d36c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5809823663a4cc5832def7f0d4698f2" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44247 } } peers { permanent_uuid: "863769ae98e240eab0f493d61e0d36c3" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } } peers { permanent_uuid: "b6e9a6d170394d21b67b5badee1c99fe" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } } }
I20260808 03:54:23.710614 14724 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e9a6d170394d21b67b5badee1c99fe [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:54:23.714998 14743 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5809823663a4cc5832def7f0d4698f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 863769ae98e240eab0f493d61e0d36c3. Latest consensus state: current_term: 1 leader_uuid: "863769ae98e240eab0f493d61e0d36c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5809823663a4cc5832def7f0d4698f2" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44247 } } peers { permanent_uuid: "863769ae98e240eab0f493d61e0d36c3" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } } peers { permanent_uuid: "b6e9a6d170394d21b67b5badee1c99fe" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } } }
I20260808 03:54:23.715332 14743 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5809823663a4cc5832def7f0d4698f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:54:23.715937 14793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "863769ae98e240eab0f493d61e0d36c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5809823663a4cc5832def7f0d4698f2" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44247 } } peers { permanent_uuid: "863769ae98e240eab0f493d61e0d36c3" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } } peers { permanent_uuid: "b6e9a6d170394d21b67b5badee1c99fe" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } } }
I20260808 03:54:23.716248 14793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [sys.catalog]: This master's current role is: LEADER
I20260808 03:54:23.717074 14796 catalog_manager.cc:1347] Generated new cluster ID: 9a80e7dc4be745a3b420d0543a0caa0c
I20260808 03:54:23.717182 14796 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:54:23.718012 14793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "863769ae98e240eab0f493d61e0d36c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5809823663a4cc5832def7f0d4698f2" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44247 } } peers { permanent_uuid: "863769ae98e240eab0f493d61e0d36c3" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } } peers { permanent_uuid: "b6e9a6d170394d21b67b5badee1c99fe" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } } }
I20260808 03:54:23.718298 14793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3 [sys.catalog]: This master's current role is: LEADER
I20260808 03:54:23.724130 14798 mvcc.cc:205] Tried to move back new op lower bound from 7316116535908233216 to 7316116535743315968. Current Snapshot: MvccSnapshot[applied={T|T < 7316116535908233216}]
I20260808 03:54:23.741879 14743 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5809823663a4cc5832def7f0d4698f2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "863769ae98e240eab0f493d61e0d36c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5809823663a4cc5832def7f0d4698f2" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44247 } } peers { permanent_uuid: "863769ae98e240eab0f493d61e0d36c3" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33539 } } peers { permanent_uuid: "b6e9a6d170394d21b67b5badee1c99fe" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 41083 } } }
I20260808 03:54:23.742224 14743 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5809823663a4cc5832def7f0d4698f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:54:23.762287 14796 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:54:23.764325 14796 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:54:23.777571 14796 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 863769ae98e240eab0f493d61e0d36c3: Generated new TSK 0
I20260808 03:54:23.778631 14796 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:54:24.433020 14740 catalog_manager.cc:1259] Loaded cluster ID: 9a80e7dc4be745a3b420d0543a0caa0c
I20260808 03:54:24.433151 14740 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b6e9a6d170394d21b67b5badee1c99fe: loading cluster ID for follower catalog manager: success
I20260808 03:54:24.435705 14740 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b6e9a6d170394d21b67b5badee1c99fe: acquiring CA information for follower catalog manager: success
I20260808 03:54:24.437484 14740 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b6e9a6d170394d21b67b5badee1c99fe: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 03:54:24.452729 14760 catalog_manager.cc:1259] Loaded cluster ID: 9a80e7dc4be745a3b420d0543a0caa0c
I20260808 03:54:24.452809 14760 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a5809823663a4cc5832def7f0d4698f2: loading cluster ID for follower catalog manager: success
I20260808 03:54:24.455088 14760 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a5809823663a4cc5832def7f0d4698f2: acquiring CA information for follower catalog manager: success
I20260808 03:54:24.456794 14760 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a5809823663a4cc5832def7f0d4698f2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 03:54:24.900269 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 14360
I20260808 03:54:24.901901 14448 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:25.010111 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 14360
I20260808 03:54:25.036355 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 14467
I20260808 03:54:25.037456 14562 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:25.143347 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 14467
I20260808 03:54:25.169306 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 14584
I20260808 03:54:25.170413 14691 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:25.277609 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 14584
2026-08-08T03:54:25Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4903 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:54:25 dist-test-slave-jlzn krb5kdc[14827](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:54:25 dist-test-slave-jlzn krb5kdc[14827](info): set up 2 sockets
Aug 08 03:54:25 dist-test-slave-jlzn krb5kdc[14827](info): commencing operation
krb5kdc: starting...
W20260808 03:54:26.472880 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.002s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:54:26 dist-test-slave-jlzn krb5kdc[14827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:54:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:54:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:54:26.730978 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:36999
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:43409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:36999
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:54:27.065970 14850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:27.066237 14850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:27.066365 14850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:27.073431 14850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:54:27.073551 14850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:27.073621 14850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:27.073684 14850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:54:27.073729 14850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:54:27.083168 14850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:43409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:36999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:36999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:27.084450 14850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:27.085698 14850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:27.096750 14858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:27.096939 14859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:27.097059 14862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:27.097903 14850 server_base.cc:1034] running on GCE node
I20260808 03:54:27.098688 14850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:27.100011 14850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:27.101233 14850 hybrid_clock.cc:648] HybridClock initialized: now 1786161267101179 us; error 38 us; skew 500 ppm
I20260808 03:54:27.101608 14850 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:27 dist-test-slave-jlzn krb5kdc[14827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161267, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:27.104645 14850 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:54:27.106042 14850 webserver.cc:460] Webserver started at http://127.26.146.190:36875/ using document root <none> and password file <none>
I20260808 03:54:27.106735 14850 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:27.106858 14850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:54:27.107154 14850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:54:27.110134 14850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "ab5575012acb4430a91c856bdc09055f"
format_stamp: "Formatted at 2026-08-08 03:54:27 on dist-test-slave-jlzn"
server_key: "f82076971a4c22132ebf6f87f7960ef8"
server_key_iv: "ba4b27866c3afbe1b920b4c2e039eab2"
server_key_version: "encryptionkey@0"
I20260808 03:54:27.110908 14850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "ab5575012acb4430a91c856bdc09055f"
format_stamp: "Formatted at 2026-08-08 03:54:27 on dist-test-slave-jlzn"
server_key: "f82076971a4c22132ebf6f87f7960ef8"
server_key_iv: "ba4b27866c3afbe1b920b4c2e039eab2"
server_key_version: "encryptionkey@0"
I20260808 03:54:27.116178 14850 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 03:54:27.119549 14878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:27.120905 14850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:54:27.121074 14850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "ab5575012acb4430a91c856bdc09055f"
format_stamp: "Formatted at 2026-08-08 03:54:27 on dist-test-slave-jlzn"
server_key: "f82076971a4c22132ebf6f87f7960ef8"
server_key_iv: "ba4b27866c3afbe1b920b4c2e039eab2"
server_key_version: "encryptionkey@0"
I20260808 03:54:27.121253 14850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:27.137902 14850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:27.141968 14850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:27.142226 14850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:27.159855 14850 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:36999
I20260808 03:54:27.159891 14951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:36999 every 8 connection(s)
I20260808 03:54:27.161170 14850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:54:27.164860 14952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:54:27.165339 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 14850
I20260808 03:54:27.165535 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:54:27.166216 27210 external_mini_cluster.cc:1428] Setting key d20a5cbd30660839049545adddbc24d2
I20260808 03:54:27.178583 14952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab5575012acb4430a91c856bdc09055f: Bootstrap starting.
Aug 08 03:54:27 dist-test-slave-jlzn krb5kdc[14827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:54:27.181602 14952 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab5575012acb4430a91c856bdc09055f: Neither blocks nor log segments found. Creating new log.
I20260808 03:54:27.182780 14952 log.cc:824] T 00000000000000000000000000000000 P ab5575012acb4430a91c856bdc09055f: Log is configured to *not* fsync() on all Append() calls
I20260808 03:54:27.186349 14952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab5575012acb4430a91c856bdc09055f: No bootstrap required, opened a new log
I20260808 03:54:27.191248 14956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:27.167958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43222 (local address 127.26.146.190:36999)
0808 03:54:27.168776 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:27.168791 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:27.168839 (+ 48us) server_negotiation.cc:407] Connection header received
0808 03:54:27.171147 (+ 2308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:27.171176 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:27.171689 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:27.172168 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:27.173508 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:27.175491 (+ 1983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:27.176983 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:27.177992 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:27.178174 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:27.182291 (+ 4117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:27.182302 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:27.182320 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:27.182363 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:27.186243 (+ 3880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:27.187107 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:27.187115 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:27.187123 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:27.187337 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:27.187841 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:27.187849 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:27.187853 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:27.188591 (+ 738us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:27.188899 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:27.189388 (+ 489us) server_negotiation.cc:299] Negotiation successful
0808 03:54:27.189850 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":298,"threads_started":1}
I20260808 03:54:27.195253 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:54:27.196367 14952 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab5575012acb4430a91c856bdc09055f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5575012acb4430a91c856bdc09055f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36999 } }
I20260808 03:54:27.196722 14952 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab5575012acb4430a91c856bdc09055f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:54:27.196815 14952 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab5575012acb4430a91c856bdc09055f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab5575012acb4430a91c856bdc09055f, State: Initialized, Role: FOLLOWER
I20260808 03:54:27.197340 14952 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab5575012acb4430a91c856bdc09055f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5575012acb4430a91c856bdc09055f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36999 } }
I20260808 03:54:27.197615 14952 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab5575012acb4430a91c856bdc09055f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:54:27.197777 14952 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab5575012acb4430a91c856bdc09055f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:54:27.197947 14952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab5575012acb4430a91c856bdc09055f [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:54:27.200196 14952 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab5575012acb4430a91c856bdc09055f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5575012acb4430a91c856bdc09055f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36999 } }
I20260808 03:54:27.200657 14952 leader_election.cc:302] T 00000000000000000000000000000000 P ab5575012acb4430a91c856bdc09055f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab5575012acb4430a91c856bdc09055f; no voters:
I20260808 03:54:27.201228 14952 leader_election.cc:288] T 00000000000000000000000000000000 P ab5575012acb4430a91c856bdc09055f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:54:27.201357 14958 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab5575012acb4430a91c856bdc09055f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:54:27.202032 14958 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab5575012acb4430a91c856bdc09055f [term 1 LEADER]: Becoming Leader. State: Replica: ab5575012acb4430a91c856bdc09055f, State: Running, Role: LEADER
I20260808 03:54:27.202504 14958 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab5575012acb4430a91c856bdc09055f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5575012acb4430a91c856bdc09055f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36999 } }
I20260808 03:54:27.203150 14952 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab5575012acb4430a91c856bdc09055f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:54:27.205488 14959 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab5575012acb4430a91c856bdc09055f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab5575012acb4430a91c856bdc09055f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5575012acb4430a91c856bdc09055f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36999 } } }
I20260808 03:54:27.205526 14961 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab5575012acb4430a91c856bdc09055f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab5575012acb4430a91c856bdc09055f. Latest consensus state: current_term: 1 leader_uuid: "ab5575012acb4430a91c856bdc09055f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5575012acb4430a91c856bdc09055f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36999 } } }
I20260808 03:54:27.205916 14961 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab5575012acb4430a91c856bdc09055f [sys.catalog]: This master's current role is: LEADER
I20260808 03:54:27.205917 14959 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab5575012acb4430a91c856bdc09055f [sys.catalog]: This master's current role is: LEADER
I20260808 03:54:27.207342 14966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:54:27.212615 14966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:54:27.221042 14966 catalog_manager.cc:1347] Generated new cluster ID: 29a7546c40d54586ba9c63eb3395c9b8
I20260808 03:54:27.221151 14966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:54:27.251843 14966 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:54:27.253748 14966 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:54:27.263025 14966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab5575012acb4430a91c856bdc09055f: Generated new TSK 0
I20260808 03:54:27.264089 14966 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:54:27.307875 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 14850
I20260808 03:54:27.309113 14947 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:27.403174 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 14850
2026-08-08T03:54:27Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2118 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:54:27 dist-test-slave-jlzn krb5kdc[14994](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:54:27 dist-test-slave-jlzn krb5kdc[14994](info): set up 2 sockets
Aug 08 03:54:27 dist-test-slave-jlzn krb5kdc[14994](info): commencing operation
krb5kdc: starting...
W20260808 03:54:28.586984 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:54:28 dist-test-slave-jlzn krb5kdc[14994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:54:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:54:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:54:28.839680 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:38815
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:38815
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:54:29.172075 15014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:29.172382 15014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:29.172523 15014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:29.179461 15014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:54:29.179574 15014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:29.179654 15014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:29.179731 15014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:54:29.179791 15014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:54:29.189213 15014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:38815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:38815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:29.190512 15014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:29.191911 15014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:29.203424 15022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:29.203197 15023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:29.203290 15025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:29.204470 15014 server_base.cc:1034] running on GCE node
I20260808 03:54:29.205207 15014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:29.206496 15014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:29.207695 15014 hybrid_clock.cc:648] HybridClock initialized: now 1786161269207637 us; error 42 us; skew 500 ppm
I20260808 03:54:29.208037 15014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:29 dist-test-slave-jlzn krb5kdc[14994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161269, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:29.213898 15014 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:54:29.215335 15014 webserver.cc:460] Webserver started at http://127.26.146.190:36151/ using document root <none> and password file <none>
I20260808 03:54:29.216013 15014 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:29.216138 15014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:54:29.216495 15014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:54:29.219476 15014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "33d149fcb1654495bdb4ae000beba703"
format_stamp: "Formatted at 2026-08-08 03:54:29 on dist-test-slave-jlzn"
server_key: "93448f31cad4011407b74119899ec8d1"
server_key_iv: "9eafa4580d7aa7d180b9591c3b1ef03d"
server_key_version: "encryptionkey@0"
I20260808 03:54:29.220259 15014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "33d149fcb1654495bdb4ae000beba703"
format_stamp: "Formatted at 2026-08-08 03:54:29 on dist-test-slave-jlzn"
server_key: "93448f31cad4011407b74119899ec8d1"
server_key_iv: "9eafa4580d7aa7d180b9591c3b1ef03d"
server_key_version: "encryptionkey@0"
I20260808 03:54:29.225145 15014 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260808 03:54:29.228317 15037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:29.229518 15014 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 03:54:29.229678 15014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "33d149fcb1654495bdb4ae000beba703"
format_stamp: "Formatted at 2026-08-08 03:54:29 on dist-test-slave-jlzn"
server_key: "93448f31cad4011407b74119899ec8d1"
server_key_iv: "9eafa4580d7aa7d180b9591c3b1ef03d"
server_key_version: "encryptionkey@0"
I20260808 03:54:29.229847 15014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:29.248404 15014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:29.252107 15014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:29.252357 15014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:29.269481 15014 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:38815
I20260808 03:54:29.269511 15117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:38815 every 8 connection(s)
I20260808 03:54:29.270771 15014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:54:29.273072 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 15014
I20260808 03:54:29.273285 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:54:29.274147 27210 external_mini_cluster.cc:1428] Setting key b96ea51be0fe2b3e2d9d6b33a3b4e2fb
I20260808 03:54:29.275091 15118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:54:29.288784 15118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703: Bootstrap starting.
Aug 08 03:54:29 dist-test-slave-jlzn krb5kdc[14994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:54:29.292310 15118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703: Neither blocks nor log segments found. Creating new log.
I20260808 03:54:29.293471 15118 log.cc:824] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703: Log is configured to *not* fsync() on all Append() calls
I20260808 03:54:29.296070 15118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703: No bootstrap required, opened a new log
I20260808 03:54:29.297658 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:29.276279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59468 (local address 127.26.146.190:38815)
0808 03:54:29.277076 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:29.277090 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:29.277136 (+ 46us) server_negotiation.cc:407] Connection header received
0808 03:54:29.279605 (+ 2469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:29.279633 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:29.280132 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:29.280561 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:29.281842 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:29.283790 (+ 1948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:29.285188 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:29.286248 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:29.286432 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:29.290813 (+ 4381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:29.290820 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:29.290833 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:29.290882 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:29.293864 (+ 2982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:29.294509 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:29.294516 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:29.294521 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:29.294689 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:29.295054 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:29.295058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:29.295060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:29.295576 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:29.295886 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:29.296330 (+ 444us) server_negotiation.cc:299] Negotiation successful
0808 03:54:29.296748 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":329,"threads_started":1}
I20260808 03:54:29.304150 15118 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d149fcb1654495bdb4ae000beba703" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38815 } }
I20260808 03:54:29.304487 15118 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:54:29.304577 15118 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d149fcb1654495bdb4ae000beba703, State: Initialized, Role: FOLLOWER
I20260808 03:54:29.305064 15118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d149fcb1654495bdb4ae000beba703" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38815 } }
I20260808 03:54:29.305284 15118 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:54:29.305403 15118 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:54:29.305549 15118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:54:29.307699 15118 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d149fcb1654495bdb4ae000beba703" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38815 } }
I20260808 03:54:29.308210 15118 leader_election.cc:302] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33d149fcb1654495bdb4ae000beba703; no voters:
I20260808 03:54:29.308832 15118 leader_election.cc:288] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:54:29.308928 15124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:54:29.309588 15124 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 1 LEADER]: Becoming Leader. State: Replica: 33d149fcb1654495bdb4ae000beba703, State: Running, Role: LEADER
I20260808 03:54:29.310133 15124 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d149fcb1654495bdb4ae000beba703" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38815 } }
I20260808 03:54:29.310698 15118 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:54:29.312757 15127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33d149fcb1654495bdb4ae000beba703. Latest consensus state: current_term: 1 leader_uuid: "33d149fcb1654495bdb4ae000beba703" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d149fcb1654495bdb4ae000beba703" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38815 } } }
I20260808 03:54:29.313073 15127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [sys.catalog]: This master's current role is: LEADER
I20260808 03:54:29.312848 15126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33d149fcb1654495bdb4ae000beba703" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d149fcb1654495bdb4ae000beba703" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38815 } } }
I20260808 03:54:29.313205 15126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [sys.catalog]: This master's current role is: LEADER
I20260808 03:54:29.313987 15133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:54:29.319762 15133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:54:29.328104 15133 catalog_manager.cc:1347] Generated new cluster ID: ab1ef5b242444f3896007b46d519ed8a
I20260808 03:54:29.328210 15133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:54:29.339636 15133 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:54:29.341012 15133 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:54:29.349992 15133 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703: Generated new TSK 0
I20260808 03:54:29.350948 15133 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
I20260808 03:54:29.417704 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:38815
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:54:29.760919 15159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:29.761189 15159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:29.761291 15159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:29.768363 15159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:29.768467 15159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:29.768595 15159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:54:29.778017 15159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:38815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:29.779188 15159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:29.780500 15159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:29.793109 15170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:29.793169 15174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:29.793210 15171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:29.793742 15159 server_base.cc:1034] running on GCE node
I20260808 03:54:29.794323 15159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:29.795275 15159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:29.796470 15159 hybrid_clock.cc:648] HybridClock initialized: now 1786161269796425 us; error 33 us; skew 500 ppm
I20260808 03:54:29.796772 15159 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:29 dist-test-slave-jlzn krb5kdc[14994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161269, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:29.799829 15159 init.cc:377] Logged in from keytab as kudu/127.26.146.129@KRBTEST.COM (short username kudu)
I20260808 03:54:29.802026 15159 webserver.cc:460] Webserver started at http://127.26.146.129:46613/ using document root <none> and password file <none>
I20260808 03:54:29.802654 15159 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:29.802768 15159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:54:29.803047 15159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:54:29.805855 15159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "c90351ca171c4990bbd8672aa972a90a"
format_stamp: "Formatted at 2026-08-08 03:54:29 on dist-test-slave-jlzn"
server_key: "c083362189564de86d89efce478fe81c"
server_key_iv: "c2b261e63244aab02fef66e99512a175"
server_key_version: "encryptionkey@0"
I20260808 03:54:29.806581 15159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "c90351ca171c4990bbd8672aa972a90a"
format_stamp: "Formatted at 2026-08-08 03:54:29 on dist-test-slave-jlzn"
server_key: "c083362189564de86d89efce478fe81c"
server_key_iv: "c2b261e63244aab02fef66e99512a175"
server_key_version: "encryptionkey@0"
I20260808 03:54:29.811473 15159 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 03:54:29.814996 15182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:29.816409 15159 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 03:54:29.816572 15159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "c90351ca171c4990bbd8672aa972a90a"
format_stamp: "Formatted at 2026-08-08 03:54:29 on dist-test-slave-jlzn"
server_key: "c083362189564de86d89efce478fe81c"
server_key_iv: "c2b261e63244aab02fef66e99512a175"
server_key_version: "encryptionkey@0"
I20260808 03:54:29.816740 15159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:29.845223 15159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:29.849439 15159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:29.849710 15159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:29.850615 15159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:54:29.851997 15159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:54:29.852072 15159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:29.852190 15159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:54:29.852257 15159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:29.882274 15159 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:44607
I20260808 03:54:29.882315 15348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:44607 every 8 connection(s)
I20260808 03:54:29.883507 15159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:54:29.886955 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 15159
I20260808 03:54:29.887183 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:54:29.888063 27210 external_mini_cluster.cc:1428] Setting key eaa91c0ba37c67c247a3c5e46da5c236
Aug 08 03:54:29 dist-test-slave-jlzn krb5kdc[14994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161269, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:54:29.903360 15352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:29.886077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:54683 (local address 127.26.146.190:38815)
0808 03:54:29.886527 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:29.886531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:29.887697 (+ 1166us) server_negotiation.cc:407] Connection header received
0808 03:54:29.889804 (+ 2107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:29.889808 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:29.889913 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:29.890015 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:29.891554 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:29.892289 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:29.893917 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:29.894313 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:29.894424 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:29.897725 (+ 3301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:29.897730 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:29.897733 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:29.897764 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:29.900387 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:29.901103 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:29.901108 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:29.901110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:29.901237 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:29.901651 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:29.901655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:29.901657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:29.901932 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:29.902077 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:29.902765 (+ 688us) server_negotiation.cc:299] Negotiation successful
0808 03:54:29.902994 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":206,"threads_started":1}
I20260808 03:54:29.904349 15353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:29.886676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.129:54683)
0808 03:54:29.887530 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:29.887579 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:29.889522 (+ 1943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:29.890287 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:29.890295 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:29.890807 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:29.891372 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:29.891391 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:29.892440 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:29.892444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:29.893769 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:29.893784 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:29.894527 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:29.894534 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:29.894767 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:29.895377 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:29.895400 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:29.897523 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:29.900541 (+ 3018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:29.900550 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:29.900567 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:29.900970 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:29.901344 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:29.901349 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:29.901352 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:29.901542 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:29.902065 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:29.902073 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:29.902395 (+ 322us) client_negotiation.cc:769] Sending connection context
0808 03:54:29.902640 (+ 245us) client_negotiation.cc:240] Negotiation successful
0808 03:54:29.903164 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":424,"threads_started":1}
I20260808 03:54:29.905925 15349 heartbeater.cc:347] Connected to a master server at 127.26.146.190:38815
I20260808 03:54:29.906253 15349 heartbeater.cc:464] Registering TS with master...
I20260808 03:54:29.907064 15349 heartbeater.cc:511] Master 127.26.146.190:38815 requested a full tablet report, sending...
I20260808 03:54:29.909545 15068 ts_manager.cc:194] Registered new tserver with Master: c90351ca171c4990bbd8672aa972a90a (127.26.146.129:44607)
I20260808 03:54:29.911540 15068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.129@KRBTEST.COM'} at 127.26.146.129:54683
WARNING: no policy specified for kudu/127.26.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
I20260808 03:54:29.949296 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:38815
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:54:30.269644 15356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:30.269915 15356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:30.270035 15356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:30.278285 15356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:30.278388 15356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:30.278548 15356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:54:30.287988 15356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:38815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:30.289182 15356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:30.290408 15356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:30.302964 15364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:30.303025 15368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:30.303059 15365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:30.304206 15356 server_base.cc:1034] running on GCE node
I20260808 03:54:30.304728 15356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:30.305534 15356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:30.306720 15356 hybrid_clock.cc:648] HybridClock initialized: now 1786161270306666 us; error 44 us; skew 500 ppm
I20260808 03:54:30.306965 15356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:30 dist-test-slave-jlzn krb5kdc[14994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161270, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:30.310111 15356 init.cc:377] Logged in from keytab as kudu/127.26.146.130@KRBTEST.COM (short username kudu)
I20260808 03:54:30.311499 15356 webserver.cc:460] Webserver started at http://127.26.146.130:38259/ using document root <none> and password file <none>
I20260808 03:54:30.312191 15356 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:30.312309 15356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:54:30.312614 15356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:54:30.315460 15356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "303151904f2a433da55e682e01bf35a8"
format_stamp: "Formatted at 2026-08-08 03:54:30 on dist-test-slave-jlzn"
server_key: "4dc4947639b416a64714e5ea60c6aacd"
server_key_iv: "a49cb8d946a3dfdabd5b55cbc3e4d388"
server_key_version: "encryptionkey@0"
I20260808 03:54:30.316289 15356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "303151904f2a433da55e682e01bf35a8"
format_stamp: "Formatted at 2026-08-08 03:54:30 on dist-test-slave-jlzn"
server_key: "4dc4947639b416a64714e5ea60c6aacd"
server_key_iv: "a49cb8d946a3dfdabd5b55cbc3e4d388"
server_key_version: "encryptionkey@0"
I20260808 03:54:30.321213 15356 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260808 03:54:30.324579 15381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:30.325876 15356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:54:30.326022 15356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "303151904f2a433da55e682e01bf35a8"
format_stamp: "Formatted at 2026-08-08 03:54:30 on dist-test-slave-jlzn"
server_key: "4dc4947639b416a64714e5ea60c6aacd"
server_key_iv: "a49cb8d946a3dfdabd5b55cbc3e4d388"
server_key_version: "encryptionkey@0"
I20260808 03:54:30.326200 15356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:30.349907 15356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:30.353909 15356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:30.354146 15356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:30.355072 15356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:54:30.356356 15356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:54:30.356433 15356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:30.356541 15356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:54:30.356585 15356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:30.386198 15356 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:40337
I20260808 03:54:30.386245 15546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:40337 every 8 connection(s)
I20260808 03:54:30.387547 15356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:54:30.393792 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 15356
I20260808 03:54:30.393972 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:54:30.394423 27210 external_mini_cluster.cc:1428] Setting key 67eebe5c139e3c8c6d3ecfc04aec80e7
Aug 08 03:54:30 dist-test-slave-jlzn krb5kdc[14994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161270, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:54:30.407348 15352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:30.390265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:55537 (local address 127.26.146.190:38815)
0808 03:54:30.390437 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:30.390442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:30.391594 (+ 1152us) server_negotiation.cc:407] Connection header received
0808 03:54:30.393437 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:30.393443 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:30.393538 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:30.393639 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:30.395416 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:30.396413 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:30.397930 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:30.398363 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:30.398473 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:30.401914 (+ 3441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:30.401919 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:30.401922 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:30.401955 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:30.404486 (+ 2531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:30.405169 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:30.405173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:30.405176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:30.405310 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:30.405726 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:30.405730 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:30.405732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:30.406019 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:30.406156 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:30.406917 (+ 761us) server_negotiation.cc:299] Negotiation successful
0808 03:54:30.407100 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 03:54:30.408823 15550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:30.390682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.130:55537)
0808 03:54:30.391400 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:30.391442 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:30.393154 (+ 1712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:30.393882 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:30.393890 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:30.394511 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:30.395239 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:30.395263 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:30.396565 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:30.396574 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:30.397785 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:30.397797 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:30.398584 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:30.398592 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:30.398807 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:30.399428 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:30.399450 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:30.401703 (+ 2253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:30.404629 (+ 2926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:30.404638 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:30.404654 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:30.405057 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:30.405415 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:30.405420 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:30.405424 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:30.405618 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:30.406141 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:30.406148 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:30.406507 (+ 359us) client_negotiation.cc:769] Sending connection context
0808 03:54:30.406822 (+ 315us) client_negotiation.cc:240] Negotiation successful
0808 03:54:30.407337 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":279,"threads_started":1}
I20260808 03:54:30.410257 15547 heartbeater.cc:347] Connected to a master server at 127.26.146.190:38815
I20260808 03:54:30.410636 15547 heartbeater.cc:464] Registering TS with master...
I20260808 03:54:30.411343 15547 heartbeater.cc:511] Master 127.26.146.190:38815 requested a full tablet report, sending...
I20260808 03:54:30.412911 15068 ts_manager.cc:194] Registered new tserver with Master: 303151904f2a433da55e682e01bf35a8 (127.26.146.130:40337)
I20260808 03:54:30.413748 15068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.130@KRBTEST.COM'} at 127.26.146.130:55537
WARNING: no policy specified for kudu/127.26.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
I20260808 03:54:30.454393 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:38815
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:54:30.777649 15561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:30.777956 15561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:30.778090 15561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:30.785281 15561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:30.785415 15561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:30.785566 15561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:54:30.795277 15561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:38815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:30.796655 15561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:30.797961 15561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:30.811070 15578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:30.811102 15580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:30.811137 15574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:30.817410 15561 server_base.cc:1034] running on GCE node
I20260808 03:54:30.818122 15561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:30.819123 15561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:30.820333 15561 hybrid_clock.cc:648] HybridClock initialized: now 1786161270820276 us; error 51 us; skew 500 ppm
I20260808 03:54:30.820677 15561 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:30 dist-test-slave-jlzn krb5kdc[14994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161270, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:30.823936 15561 init.cc:377] Logged in from keytab as kudu/127.26.146.131@KRBTEST.COM (short username kudu)
I20260808 03:54:30.825467 15561 webserver.cc:460] Webserver started at http://127.26.146.131:35781/ using document root <none> and password file <none>
I20260808 03:54:30.826174 15561 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:30.826295 15561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:54:30.826587 15561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:54:30.829615 15561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "65b52d7e7d524c3983380747aaefc15c"
format_stamp: "Formatted at 2026-08-08 03:54:30 on dist-test-slave-jlzn"
server_key: "e1a1611eff7a5b97e6a36ba64618f124"
server_key_iv: "62ab018894e8bb06ad5f2a4fe520c636"
server_key_version: "encryptionkey@0"
I20260808 03:54:30.830432 15561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "65b52d7e7d524c3983380747aaefc15c"
format_stamp: "Formatted at 2026-08-08 03:54:30 on dist-test-slave-jlzn"
server_key: "e1a1611eff7a5b97e6a36ba64618f124"
server_key_iv: "62ab018894e8bb06ad5f2a4fe520c636"
server_key_version: "encryptionkey@0"
I20260808 03:54:30.835522 15561 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 03:54:30.838912 15592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:30.840303 15561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:54:30.840462 15561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "65b52d7e7d524c3983380747aaefc15c"
format_stamp: "Formatted at 2026-08-08 03:54:30 on dist-test-slave-jlzn"
server_key: "e1a1611eff7a5b97e6a36ba64618f124"
server_key_iv: "62ab018894e8bb06ad5f2a4fe520c636"
server_key_version: "encryptionkey@0"
I20260808 03:54:30.840670 15561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:30.867873 15561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:30.873536 15561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:30.873804 15561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:30.874742 15561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:54:30.876130 15561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:54:30.876212 15561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:30.876305 15561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:54:30.876348 15561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:30.905922 15561 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:34861
I20260808 03:54:30.905964 15765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:34861 every 8 connection(s)
I20260808 03:54:30.907151 15561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:54:30.907924 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 15561
I20260808 03:54:30.908113 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:54:30.908737 27210 external_mini_cluster.cc:1428] Setting key cb8b4b34d55071bdcc89418c6c32db0e
I20260808 03:54:30.915125 15349 heartbeater.cc:503] Master 127.26.146.190:38815 was elected leader, sending a full tablet report...
Aug 08 03:54:30 dist-test-slave-jlzn krb5kdc[14994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161270, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:54:30.927386 15770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:30.909780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:33099 (local address 127.26.146.190:38815)
0808 03:54:30.910340 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:30.910346 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:30.911332 (+ 986us) server_negotiation.cc:407] Connection header received
0808 03:54:30.913359 (+ 2027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:30.913364 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:30.913431 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:30.913525 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:30.914960 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:30.915987 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:30.917649 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:30.918050 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:30.918156 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:30.921552 (+ 3396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:30.921557 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:30.921560 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:30.921594 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:30.924316 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:30.925060 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:30.925065 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:30.925068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:30.925193 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:30.925626 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:30.925631 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:30.925635 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:30.925908 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:30.926052 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:30.926781 (+ 729us) server_negotiation.cc:299] Negotiation successful
0808 03:54:30.927020 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":270,"threads_started":1}
I20260808 03:54:30.928367 15774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:30.910355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.131:33099)
0808 03:54:30.911156 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:30.911207 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:30.913095 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:30.913738 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:30.913747 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:30.914235 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:30.914804 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:30.914822 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:30.916133 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:30.916137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:30.917503 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:30.917529 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:30.918258 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:30.918266 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:30.918473 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:30.919125 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:30.919147 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:30.921348 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:30.924464 (+ 3116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:30.924473 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:30.924499 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:30.924937 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:30.925300 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:30.925305 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:30.925309 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:30.925514 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:30.926045 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:30.926052 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:30.926389 (+ 337us) client_negotiation.cc:769] Sending connection context
0808 03:54:30.926651 (+ 262us) client_negotiation.cc:240] Negotiation successful
0808 03:54:30.927201 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":309,"threads_started":1}
I20260808 03:54:30.929733 15767 heartbeater.cc:347] Connected to a master server at 127.26.146.190:38815
I20260808 03:54:30.930032 15767 heartbeater.cc:464] Registering TS with master...
I20260808 03:54:30.930804 15767 heartbeater.cc:511] Master 127.26.146.190:38815 requested a full tablet report, sending...
I20260808 03:54:30.932230 15068 ts_manager.cc:194] Registered new tserver with Master: 65b52d7e7d524c3983380747aaefc15c (127.26.146.131:34861)
I20260808 03:54:30.933023 15068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.131@KRBTEST.COM'} at 127.26.146.131:33099
I20260808 03:54:30.936586 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:54:30.953230 15770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:30.941087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59478 (local address 127.26.146.190:38815)
0808 03:54:30.941284 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:30.941289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:30.941705 (+ 416us) server_negotiation.cc:407] Connection header received
0808 03:54:30.941861 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:30.941867 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:30.941933 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:30.942011 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:30.943363 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:30.944223 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:30.945511 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:30.945903 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:30.946008 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:30.948339 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:30.948344 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:30.948347 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:30.948377 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:30.950668 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:30.951313 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:30.951319 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:30.951323 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:30.951441 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:30.951825 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:30.951831 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:30.951834 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:30.952143 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:30.952367 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:30.952712 (+ 345us) server_negotiation.cc:299] Negotiation successful
0808 03:54:30.952922 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 03:54:30.956997 15068 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:54:30.963755 15068 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:54:30.992724 15791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:30.975576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:34861 (local address 127.0.0.1:54216)
0808 03:54:30.983750 (+ 8174us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:30.983783 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:30.984122 (+ 339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:30.985230 (+ 1108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:30.985238 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:30.985280 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:54:30.985788 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:30.985802 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:30.987924 (+ 2122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:30.987930 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:30.989628 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:30.989644 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:30.991697 (+ 2053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:30.991705 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:30.991893 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:30.991946 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 03:54:30.992073 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 03:54:30.992260 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7982,"mutex_wait_us":19,"thread_start_us":8392,"threads_started":1}
I20260808 03:54:30.993024 15793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:30.976538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60376 (local address 127.26.146.129:44607)
0808 03:54:30.983391 (+ 6853us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:30.983400 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:30.984354 (+ 954us) server_negotiation.cc:407] Connection header received
0808 03:54:30.984641 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:30.984649 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:30.984873 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:30.985024 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:54:30.986564 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:30.988078 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:30.990532 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:30.991979 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:30.992169 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:30.992330 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:30.992519 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 03:54:30.992702 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6693,"thread_start_us":6587,"threads_started":1}
I20260808 03:54:30.993331 15792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:30.975919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54216 (local address 127.26.146.131:34861)
0808 03:54:30.983095 (+ 7176us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:30.983115 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:30.983916 (+ 801us) server_negotiation.cc:407] Connection header received
0808 03:54:30.984415 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:30.984423 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:30.984676 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:30.984837 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:54:30.986073 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:30.987725 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:30.989792 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:30.991175 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:30.991361 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:30.991562 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:30.992851 (+ 1289us) server_negotiation.cc:299] Negotiation successful
0808 03:54:30.993015 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7000,"thread_start_us":6842,"threads_started":1}
I20260808 03:54:30.994017 15789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:30.975705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:44607 (local address 127.0.0.1:60376)
0808 03:54:30.984250 (+ 8545us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:30.984266 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:30.984456 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:30.985295 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:30.985299 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:30.985318 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:54:30.986362 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:30.986377 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:30.988217 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:30.988222 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:30.990339 (+ 2117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:30.990357 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:30.992169 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:30.992175 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:30.992345 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:30.992377 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 03:54:30.992462 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 03:54:30.992603 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8411,"mutex_wait_us":465,"thread_start_us":7732,"threads_started":1}
I20260808 03:54:30.994338 15796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:30.985158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:40337 (local address 127.0.0.1:57104)
0808 03:54:30.986465 (+ 1307us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:30.986481 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:30.986586 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:30.987272 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:30.987277 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:30.987303 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:54:30.987725 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:30.987737 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:30.989906 (+ 2169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:30.989909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:30.991798 (+ 1889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:30.991838 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:30.993555 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:30.993561 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:30.993735 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:30.993762 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 03:54:30.993842 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 03:54:30.993967 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1183,"thread_start_us":995,"threads_started":1}
I20260808 03:54:30.994360 15795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:30.984581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57104 (local address 127.26.146.130:40337)
0808 03:54:30.986000 (+ 1419us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:30.986008 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:30.986553 (+ 545us) server_negotiation.cc:407] Connection header received
0808 03:54:30.986709 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:30.986714 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:30.986866 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:30.987014 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:54:30.987971 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:30.989453 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:30.991984 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:30.993378 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:30.993541 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:30.993695 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:30.993893 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 03:54:30.994047 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1258,"thread_start_us":1121,"threads_started":1}
I20260808 03:54:30.997678 15443 tablet_service.cc:1467] Processing CreateTablet for tablet cd11af7ee04a485ea82048caadeff2aa (DEFAULT_TABLE table=test-table [id=766022aa78d34040b3c7d8d041614301]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:54:30.997695 15657 tablet_service.cc:1467] Processing CreateTablet for tablet cd11af7ee04a485ea82048caadeff2aa (DEFAULT_TABLE table=test-table [id=766022aa78d34040b3c7d8d041614301]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:54:30.997896 15249 tablet_service.cc:1467] Processing CreateTablet for tablet cd11af7ee04a485ea82048caadeff2aa (DEFAULT_TABLE table=test-table [id=766022aa78d34040b3c7d8d041614301]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:54:30.998895 15443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd11af7ee04a485ea82048caadeff2aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:54:30.999109 15249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd11af7ee04a485ea82048caadeff2aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:54:30.999119 15657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd11af7ee04a485ea82048caadeff2aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:54:31.010762 15797 tablet_bootstrap.cc:492] T cd11af7ee04a485ea82048caadeff2aa P 303151904f2a433da55e682e01bf35a8: Bootstrap starting.
I20260808 03:54:31.013115 15802 tablet_bootstrap.cc:492] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c: Bootstrap starting.
I20260808 03:54:31.013912 15801 tablet_bootstrap.cc:492] T cd11af7ee04a485ea82048caadeff2aa P c90351ca171c4990bbd8672aa972a90a: Bootstrap starting.
I20260808 03:54:31.015424 15797 tablet_bootstrap.cc:654] T cd11af7ee04a485ea82048caadeff2aa P 303151904f2a433da55e682e01bf35a8: Neither blocks nor log segments found. Creating new log.
I20260808 03:54:31.016777 15797 log.cc:824] T cd11af7ee04a485ea82048caadeff2aa P 303151904f2a433da55e682e01bf35a8: Log is configured to *not* fsync() on all Append() calls
I20260808 03:54:31.017407 15801 tablet_bootstrap.cc:654] T cd11af7ee04a485ea82048caadeff2aa P c90351ca171c4990bbd8672aa972a90a: Neither blocks nor log segments found. Creating new log.
I20260808 03:54:31.018507 15801 log.cc:824] T cd11af7ee04a485ea82048caadeff2aa P c90351ca171c4990bbd8672aa972a90a: Log is configured to *not* fsync() on all Append() calls
I20260808 03:54:31.018833 15802 tablet_bootstrap.cc:654] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c: Neither blocks nor log segments found. Creating new log.
I20260808 03:54:31.019951 15797 tablet_bootstrap.cc:492] T cd11af7ee04a485ea82048caadeff2aa P 303151904f2a433da55e682e01bf35a8: No bootstrap required, opened a new log
I20260808 03:54:31.020051 15802 log.cc:824] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c: Log is configured to *not* fsync() on all Append() calls
I20260808 03:54:31.020287 15797 ts_tablet_manager.cc:1397] T cd11af7ee04a485ea82048caadeff2aa P 303151904f2a433da55e682e01bf35a8: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260808 03:54:31.021623 15801 tablet_bootstrap.cc:492] T cd11af7ee04a485ea82048caadeff2aa P c90351ca171c4990bbd8672aa972a90a: No bootstrap required, opened a new log
I20260808 03:54:31.021929 15801 ts_tablet_manager.cc:1397] T cd11af7ee04a485ea82048caadeff2aa P c90351ca171c4990bbd8672aa972a90a: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 03:54:31.022614 15802 tablet_bootstrap.cc:492] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c: No bootstrap required, opened a new log
I20260808 03:54:31.022914 15802 ts_tablet_manager.cc:1397] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260808 03:54:31.029594 15802 raft_consensus.cc:348] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } }
I20260808 03:54:31.029961 15802 raft_consensus.cc:374] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:54:31.030082 15802 raft_consensus.cc:729] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65b52d7e7d524c3983380747aaefc15c, State: Initialized, Role: FOLLOWER
I20260808 03:54:31.030665 15802 consensus_queue.cc:260] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } }
I20260808 03:54:31.030895 15797 raft_consensus.cc:348] T cd11af7ee04a485ea82048caadeff2aa P 303151904f2a433da55e682e01bf35a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } }
I20260808 03:54:31.031356 15797 raft_consensus.cc:374] T cd11af7ee04a485ea82048caadeff2aa P 303151904f2a433da55e682e01bf35a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:54:31.031483 15797 raft_consensus.cc:729] T cd11af7ee04a485ea82048caadeff2aa P 303151904f2a433da55e682e01bf35a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 303151904f2a433da55e682e01bf35a8, State: Initialized, Role: FOLLOWER
I20260808 03:54:31.031724 15767 heartbeater.cc:503] Master 127.26.146.190:38815 was elected leader, sending a full tablet report...
I20260808 03:54:31.032217 15797 consensus_queue.cc:260] T cd11af7ee04a485ea82048caadeff2aa P 303151904f2a433da55e682e01bf35a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } }
I20260808 03:54:31.032145 15801 raft_consensus.cc:348] T cd11af7ee04a485ea82048caadeff2aa P c90351ca171c4990bbd8672aa972a90a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } }
I20260808 03:54:31.032527 15801 raft_consensus.cc:374] T cd11af7ee04a485ea82048caadeff2aa P c90351ca171c4990bbd8672aa972a90a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:54:31.032665 15801 raft_consensus.cc:729] T cd11af7ee04a485ea82048caadeff2aa P c90351ca171c4990bbd8672aa972a90a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c90351ca171c4990bbd8672aa972a90a, State: Initialized, Role: FOLLOWER
I20260808 03:54:31.033317 15801 consensus_queue.cc:260] T cd11af7ee04a485ea82048caadeff2aa P c90351ca171c4990bbd8672aa972a90a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } }
I20260808 03:54:31.035219 15801 ts_tablet_manager.cc:1428] T cd11af7ee04a485ea82048caadeff2aa P c90351ca171c4990bbd8672aa972a90a: Time spent starting tablet: real 0.013s user 0.004s sys 0.008s
I20260808 03:54:31.036204 15547 heartbeater.cc:503] Master 127.26.146.190:38815 was elected leader, sending a full tablet report...
I20260808 03:54:31.036789 15797 ts_tablet_manager.cc:1428] T cd11af7ee04a485ea82048caadeff2aa P 303151904f2a433da55e682e01bf35a8: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260808 03:54:31.036929 15802 ts_tablet_manager.cc:1428] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c: Time spent starting tablet: real 0.014s user 0.009s sys 0.000s
I20260808 03:54:31.093091 15808 raft_consensus.cc:482] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:54:31.093389 15808 raft_consensus.cc:504] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } }
I20260808 03:54:31.094956 15808 leader_election.cc:288] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c90351ca171c4990bbd8672aa972a90a (127.26.146.129:44607), 303151904f2a433da55e682e01bf35a8 (127.26.146.130:40337)
I20260808 03:54:31.101169 15793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:31.095347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:40175 (local address 127.26.146.129:44607)
0808 03:54:31.095704 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:31.095710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:31.095734 (+ 24us) server_negotiation.cc:407] Connection header received
0808 03:54:31.095925 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:31.095931 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:31.096006 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:31.096070 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:54:31.096670 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:31.097596 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:31.099316 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:31.100401 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:31.100514 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:31.100569 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:31.100880 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 03:54:31.100960 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 03:54:31.101257 15774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:31.095208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:44607 (local address 127.26.146.131:40175)
0808 03:54:31.095563 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:31.095583 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:31.095719 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:31.096292 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:31.096297 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:31.096327 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:54:31.096512 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:31.096523 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:31.097746 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:31.097751 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:31.099164 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:31.099178 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:31.100538 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:31.100544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:31.100737 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:31.100771 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 03:54:31.100865 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 03:54:31.100993 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211}
I20260808 03:54:31.101996 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:31.095560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:40337 (local address 127.26.146.131:33029)
0808 03:54:31.096233 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:31.096247 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:31.096353 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:31.096670 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:31.096675 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:31.096700 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:54:31.097054 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:31.097066 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:31.098149 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:31.098153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:31.099939 (+ 1786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:31.099954 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:31.101301 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:31.101308 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:31.101462 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:31.101491 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 03:54:31.101635 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 03:54:31.101722 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":369,"threads_started":1}
I20260808 03:54:31.102027 15795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:31.095977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:33029 (local address 127.26.146.130:40337)
0808 03:54:31.096182 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:31.096187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:31.096327 (+ 140us) server_negotiation.cc:407] Connection header received
0808 03:54:31.096475 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:31.096479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:31.096549 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:31.096643 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:54:31.097211 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:31.097994 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:31.100098 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:31.101085 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:31.101230 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:31.101298 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:31.101688 (+ 390us) server_negotiation.cc:299] Negotiation successful
0808 03:54:31.101842 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 03:54:31.102386 15283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd11af7ee04a485ea82048caadeff2aa" candidate_uuid: "65b52d7e7d524c3983380747aaefc15c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c90351ca171c4990bbd8672aa972a90a" is_pre_election: true
I20260808 03:54:31.102941 15283 raft_consensus.cc:2454] T cd11af7ee04a485ea82048caadeff2aa P c90351ca171c4990bbd8672aa972a90a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65b52d7e7d524c3983380747aaefc15c in term 0.
I20260808 03:54:31.103008 15479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd11af7ee04a485ea82048caadeff2aa" candidate_uuid: "65b52d7e7d524c3983380747aaefc15c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "303151904f2a433da55e682e01bf35a8" is_pre_election: true
I20260808 03:54:31.103478 15479 raft_consensus.cc:2454] T cd11af7ee04a485ea82048caadeff2aa P 303151904f2a433da55e682e01bf35a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65b52d7e7d524c3983380747aaefc15c in term 0.
I20260808 03:54:31.103535 15599 leader_election.cc:302] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65b52d7e7d524c3983380747aaefc15c, c90351ca171c4990bbd8672aa972a90a; no voters:
I20260808 03:54:31.103982 15808 raft_consensus.cc:2781] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:54:31.104146 15808 raft_consensus.cc:482] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:54:31.104264 15808 raft_consensus.cc:3037] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:54:31.106400 15808 raft_consensus.cc:504] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } }
I20260808 03:54:31.106968 15808 leader_election.cc:288] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c [CANDIDATE]: Term 1 election: Requested vote from peers c90351ca171c4990bbd8672aa972a90a (127.26.146.129:44607), 303151904f2a433da55e682e01bf35a8 (127.26.146.130:40337)
I20260808 03:54:31.107327 15283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd11af7ee04a485ea82048caadeff2aa" candidate_uuid: "65b52d7e7d524c3983380747aaefc15c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c90351ca171c4990bbd8672aa972a90a"
I20260808 03:54:31.107436 15479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd11af7ee04a485ea82048caadeff2aa" candidate_uuid: "65b52d7e7d524c3983380747aaefc15c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "303151904f2a433da55e682e01bf35a8"
I20260808 03:54:31.107513 15283 raft_consensus.cc:3037] T cd11af7ee04a485ea82048caadeff2aa P c90351ca171c4990bbd8672aa972a90a [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:54:31.107625 15479 raft_consensus.cc:3037] T cd11af7ee04a485ea82048caadeff2aa P 303151904f2a433da55e682e01bf35a8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:54:31.109710 15283 raft_consensus.cc:2454] T cd11af7ee04a485ea82048caadeff2aa P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65b52d7e7d524c3983380747aaefc15c in term 1.
I20260808 03:54:31.109822 15479 raft_consensus.cc:2454] T cd11af7ee04a485ea82048caadeff2aa P 303151904f2a433da55e682e01bf35a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65b52d7e7d524c3983380747aaefc15c in term 1.
I20260808 03:54:31.110118 15599 leader_election.cc:302] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65b52d7e7d524c3983380747aaefc15c, c90351ca171c4990bbd8672aa972a90a; no voters:
I20260808 03:54:31.110374 15808 raft_consensus.cc:2781] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:54:31.110816 15808 raft_consensus.cc:686] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c [term 1 LEADER]: Becoming Leader. State: Replica: 65b52d7e7d524c3983380747aaefc15c, State: Running, Role: LEADER
I20260808 03:54:31.111295 15808 consensus_queue.cc:237] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } }
I20260808 03:54:31.116158 15065 catalog_manager.cc:5495] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c reported cstate change: term changed from 0 to 1, leader changed from <none> to 65b52d7e7d524c3983380747aaefc15c (127.26.146.131). New cstate: current_term: 1 leader_uuid: "65b52d7e7d524c3983380747aaefc15c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } health_report { overall_health: UNKNOWN } } }
W20260808 03:54:31.135824 15350 tablet.cc:2369] T cd11af7ee04a485ea82048caadeff2aa P c90351ca171c4990bbd8672aa972a90a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:54:31.139489 15549 tablet.cc:2369] T cd11af7ee04a485ea82048caadeff2aa P 303151904f2a433da55e682e01bf35a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:54:31.147001 15792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:31.142005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54220 (local address 127.26.146.131:34861)
0808 03:54:31.142200 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:31.142205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:31.142223 (+ 18us) server_negotiation.cc:407] Connection header received
0808 03:54:31.142304 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:31.142308 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:31.142389 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:31.142510 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:54:31.142830 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:31.143636 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:31.145018 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:31.145461 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:31.145558 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:31.146006 (+ 448us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:54:31.146507 (+ 501us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:54:31.146663 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:31.146701 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 03:54:31.146762 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 03:54:31.154678 15283 raft_consensus.cc:1261] T cd11af7ee04a485ea82048caadeff2aa P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Refusing update from remote peer 65b52d7e7d524c3983380747aaefc15c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:54:31.154721 15479 raft_consensus.cc:1261] T cd11af7ee04a485ea82048caadeff2aa P 303151904f2a433da55e682e01bf35a8 [term 1 FOLLOWER]: Refusing update from remote peer 65b52d7e7d524c3983380747aaefc15c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:54:31.155593 15815 consensus_queue.cc:1035] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:54:31.155946 15808 consensus_queue.cc:1035] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c [LEADER]: Connected to new peer: Peer: permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 03:54:31.158910 15768 tablet.cc:2369] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:54:31.170424 15821 mvcc.cc:205] Tried to move back new op lower bound from 7316116566640394240 to 7316116566475771904. Current Snapshot: MvccSnapshot[applied={T|T < 7316116566640394240}]
I20260808 03:54:31.180276 15065 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:54:31.180538 15065 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:54:31.184125 15065 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703: Sending DeleteTablet for 3 replicas of tablet cd11af7ee04a485ea82048caadeff2aa
I20260808 03:54:31.185226 15657 tablet_service.cc:1514] Processing DeleteTablet for tablet cd11af7ee04a485ea82048caadeff2aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:54:31 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:54216
I20260808 03:54:31.185355 15249 tablet_service.cc:1514] Processing DeleteTablet for tablet cd11af7ee04a485ea82048caadeff2aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:54:31 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:60376
I20260808 03:54:31.185355 15443 tablet_service.cc:1514] Processing DeleteTablet for tablet cd11af7ee04a485ea82048caadeff2aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:54:31 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:57104
I20260808 03:54:31.186446 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 15159
I20260808 03:54:31.197657 15830 tablet_replica.cc:330] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c: stopping tablet replica
I20260808 03:54:31.198760 15830 raft_consensus.cc:2229] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:54:31.200228 15830 raft_consensus.cc:2258] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:54:31.201088 15832 tablet_replica.cc:330] T cd11af7ee04a485ea82048caadeff2aa P c90351ca171c4990bbd8672aa972a90a: stopping tablet replica
I20260808 03:54:31.202320 15832 raft_consensus.cc:2229] T cd11af7ee04a485ea82048caadeff2aa P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:54:31.205010 15832 raft_consensus.cc:2258] T cd11af7ee04a485ea82048caadeff2aa P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:54:31.205796 15830 ts_tablet_manager.cc:1905] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:54:31.208361 15832 ts_tablet_manager.cc:1905] T cd11af7ee04a485ea82048caadeff2aa P c90351ca171c4990bbd8672aa972a90a: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 03:54:31 dist-test-slave-jlzn krb5kdc[14994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.129@KRBTEST.COM
I20260808 03:54:31.210883 15833 tablet_replica.cc:330] T cd11af7ee04a485ea82048caadeff2aa P 303151904f2a433da55e682e01bf35a8: stopping tablet replica
I20260808 03:54:31.211391 15833 raft_consensus.cc:2229] T cd11af7ee04a485ea82048caadeff2aa P 303151904f2a433da55e682e01bf35a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:54:31.212085 15833 raft_consensus.cc:2258] T cd11af7ee04a485ea82048caadeff2aa P 303151904f2a433da55e682e01bf35a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:54:31.214318 15833 ts_tablet_manager.cc:1905] T cd11af7ee04a485ea82048caadeff2aa P 303151904f2a433da55e682e01bf35a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:54:31.215969 15830 ts_tablet_manager.cc:1918] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:54:31.216233 15830 log.cc:1196] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/cd11af7ee04a485ea82048caadeff2aa
I20260808 03:54:31.216897 15830 ts_tablet_manager.cc:1939] T cd11af7ee04a485ea82048caadeff2aa P 65b52d7e7d524c3983380747aaefc15c: Deleting consensus metadata
I20260808 03:54:31.218775 15042 catalog_manager.cc:4841] TS 65b52d7e7d524c3983380747aaefc15c (127.26.146.131:34861): tablet cd11af7ee04a485ea82048caadeff2aa (table test-table [id=766022aa78d34040b3c7d8d041614301]) successfully deleted
I20260808 03:54:31.220865 15833 ts_tablet_manager.cc:1918] T cd11af7ee04a485ea82048caadeff2aa P 303151904f2a433da55e682e01bf35a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:54:31.221099 15833 log.cc:1196] T cd11af7ee04a485ea82048caadeff2aa P 303151904f2a433da55e682e01bf35a8: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/cd11af7ee04a485ea82048caadeff2aa
I20260808 03:54:31.221340 15832 ts_tablet_manager.cc:1918] T cd11af7ee04a485ea82048caadeff2aa P c90351ca171c4990bbd8672aa972a90a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:54:31.221582 15832 log.cc:1196] T cd11af7ee04a485ea82048caadeff2aa P c90351ca171c4990bbd8672aa972a90a: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/wal/wals/cd11af7ee04a485ea82048caadeff2aa
I20260808 03:54:31.221714 15833 ts_tablet_manager.cc:1939] T cd11af7ee04a485ea82048caadeff2aa P 303151904f2a433da55e682e01bf35a8: Deleting consensus metadata
I20260808 03:54:31.222178 15832 ts_tablet_manager.cc:1939] T cd11af7ee04a485ea82048caadeff2aa P c90351ca171c4990bbd8672aa972a90a: Deleting consensus metadata
I20260808 03:54:31.223167 15793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:31.187657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60388 (local address 127.26.146.129:44607)
0808 03:54:31.187873 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:31.187879 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:31.195433 (+ 7554us) server_negotiation.cc:407] Connection header received
0808 03:54:31.195663 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:31.195670 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:31.195773 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:31.195904 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:31.199165 (+ 3261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:31.200412 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:31.202675 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:31.203314 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:31.203410 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:31.211923 (+ 8513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:31.211937 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:31.211958 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:31.212000 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:31.216010 (+ 4010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:31.220186 (+ 4176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:31.220195 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:31.220199 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:31.220384 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:31.221000 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:31.221006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:31.221009 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:31.221521 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:31.221841 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:31.222510 (+ 669us) server_negotiation.cc:299] Negotiation successful
0808 03:54:31.222768 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 03:54:31.223511 15045 catalog_manager.cc:4841] TS 303151904f2a433da55e682e01bf35a8 (127.26.146.130:40337): tablet cd11af7ee04a485ea82048caadeff2aa (table test-table [id=766022aa78d34040b3c7d8d041614301]) successfully deleted
I20260808 03:54:31.224030 15045 catalog_manager.cc:4841] TS c90351ca171c4990bbd8672aa972a90a (127.26.146.129:44607): tablet cd11af7ee04a485ea82048caadeff2aa (table test-table [id=766022aa78d34040b3c7d8d041614301]) successfully deleted
I20260808 03:54:31.224673 15342 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:31.313987 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 15159
I20260808 03:54:31.347972 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 15356
Aug 08 03:54:31 dist-test-slave-jlzn krb5kdc[14994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.130@KRBTEST.COM
I20260808 03:54:31.364605 15795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:31.348822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57108 (local address 127.26.146.130:40337)
0808 03:54:31.349020 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:31.349026 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:31.349048 (+ 22us) server_negotiation.cc:407] Connection header received
0808 03:54:31.349209 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:31.349215 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:31.349303 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:31.349395 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:31.350734 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:31.351889 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:31.353143 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:31.353770 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:31.353894 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:31.358161 (+ 4267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:31.358179 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:31.358199 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:31.358241 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:31.361879 (+ 3638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:31.362516 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:31.362525 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:31.362529 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:31.362683 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:31.362991 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:31.362996 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:31.363000 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:31.363467 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:31.363755 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:31.364018 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 03:54:31.364240 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 03:54:31.365296 15539 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:31.453938 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 15356
I20260808 03:54:31.476004 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 15561
Aug 08 03:54:31 dist-test-slave-jlzn krb5kdc[14994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.131@KRBTEST.COM
I20260808 03:54:31.492159 15792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:31.476755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54230 (local address 127.26.146.131:34861)
0808 03:54:31.476937 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:31.476943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:31.477022 (+ 79us) server_negotiation.cc:407] Connection header received
0808 03:54:31.477239 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:31.477245 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:31.477323 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:31.477408 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:31.478709 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:31.479514 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:31.480967 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:31.481416 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:31.481529 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:31.486043 (+ 4514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:31.486057 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:31.486077 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:31.486121 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:31.489269 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:31.489956 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:31.489963 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:31.489966 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:31.490137 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:31.490500 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:31.490503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:31.490505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:31.490872 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:31.491184 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:31.491476 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 03:54:31.491753 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 03:54:31.492803 15755 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:31.614040 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 15561
I20260808 03:54:31.637957 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 15014
I20260808 03:54:31.639125 15113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:31.748044 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 15014
I20260808 03:54:31.775908 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:38815
--webserver_interface=127.26.146.190
--webserver_port=36151
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:38815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:54:32.101953 15844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:32.102246 15844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:32.102345 15844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:32.109334 15844 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:54:32.109452 15844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:32.109531 15844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:32.109611 15844 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:54:32.109671 15844 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:54:32.119248 15844 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:38815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:38815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=36151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:32.120604 15844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:32.121861 15844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:32.144663 15855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:32.144701 15856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:32.144742 15858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:32.145499 15844 server_base.cc:1034] running on GCE node
I20260808 03:54:32.146337 15844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:32.147861 15844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:32.149093 15844 hybrid_clock.cc:648] HybridClock initialized: now 1786161272149026 us; error 38 us; skew 500 ppm
I20260808 03:54:32.149483 15844 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:32 dist-test-slave-jlzn krb5kdc[14994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161272, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:32.152945 15844 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:54:32.154496 15844 webserver.cc:460] Webserver started at http://127.26.146.190:36151/ using document root <none> and password file <none>
I20260808 03:54:32.155228 15844 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:32.155368 15844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:54:32.161199 15844 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 03:54:32.164021 15866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:32.165316 15844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 03:54:32.165485 15844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "33d149fcb1654495bdb4ae000beba703"
format_stamp: "Formatted at 2026-08-08 03:54:29 on dist-test-slave-jlzn"
server_key: "93448f31cad4011407b74119899ec8d1"
server_key_iv: "9eafa4580d7aa7d180b9591c3b1ef03d"
server_key_version: "encryptionkey@0"
I20260808 03:54:32.166544 15844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:32.198359 15844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:32.202492 15844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:32.202759 15844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:32.220566 15844 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:38815
I20260808 03:54:32.220574 15957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:38815 every 8 connection(s)
I20260808 03:54:32.222043 15844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:54:32.228852 15960 sys_catalog.cc:263] Verifying existing consensus state
I20260808 03:54:32.231412 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 15844
I20260808 03:54:32.232143 15960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703: Bootstrap starting.
I20260808 03:54:32.232441 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:44607
--local_ip_for_outbound_sockets=127.26.146.129
--tserver_master_addrs=127.26.146.190:38815
--webserver_port=46613
--webserver_interface=127.26.146.129
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:54:32.254781 15960 log.cc:824] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703: Log is configured to *not* fsync() on all Append() calls
I20260808 03:54:32.271191 15960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 03:54:32.271903 15960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703: Bootstrap complete.
I20260808 03:54:32.285293 15960 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d149fcb1654495bdb4ae000beba703" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38815 } }
I20260808 03:54:32.286432 15960 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d149fcb1654495bdb4ae000beba703, State: Initialized, Role: FOLLOWER
I20260808 03:54:32.287139 15960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d149fcb1654495bdb4ae000beba703" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38815 } }
I20260808 03:54:32.287428 15960 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:54:32.287561 15960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:54:32.287734 15960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 1 FOLLOWER]: Advancing to term 2
I20260808 03:54:32.293213 15960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d149fcb1654495bdb4ae000beba703" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38815 } }
I20260808 03:54:32.293833 15960 leader_election.cc:302] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33d149fcb1654495bdb4ae000beba703; no voters:
I20260808 03:54:32.294479 15960 leader_election.cc:288] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 03:54:32.294673 15965 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 03:54:32.295861 15965 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 2 LEADER]: Becoming Leader. State: Replica: 33d149fcb1654495bdb4ae000beba703, State: Running, Role: LEADER
I20260808 03:54:32.296391 15965 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d149fcb1654495bdb4ae000beba703" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38815 } }
I20260808 03:54:32.296917 15960 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:54:32.298796 15968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33d149fcb1654495bdb4ae000beba703. Latest consensus state: current_term: 2 leader_uuid: "33d149fcb1654495bdb4ae000beba703" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d149fcb1654495bdb4ae000beba703" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38815 } } }
I20260808 03:54:32.299180 15968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [sys.catalog]: This master's current role is: LEADER
I20260808 03:54:32.299782 15966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "33d149fcb1654495bdb4ae000beba703" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d149fcb1654495bdb4ae000beba703" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38815 } } }
I20260808 03:54:32.300292 15966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [sys.catalog]: This master's current role is: LEADER
I20260808 03:54:32.307473 15974 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:54:32.315229 15974 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:54:32.316704 15974 catalog_manager.cc:1259] Loaded cluster ID: ab1ef5b242444f3896007b46d519ed8a
I20260808 03:54:32.316854 15974 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:54:32.320223 15974 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 03:54:32.338006 15974 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 03:54:32.348662 15974 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:54:32.350598 15974 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703: Loaded TSK: 0
I20260808 03:54:32.352252 15974 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 03:54:32.607930 15962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:32.608206 15962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:32.608330 15962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:32.615464 15962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:32.615584 15962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:32.615746 15962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:54:32.625730 15962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:44607
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=46613
--tserver_master_addrs=127.26.146.190:38815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:32.627029 15962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:32.628397 15962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:32.641269 15998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:32.641292 15999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:32.642385 16001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:32.642477 15962 server_base.cc:1034] running on GCE node
I20260808 03:54:32.643227 15962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:32.644438 15962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:32.645679 15962 hybrid_clock.cc:648] HybridClock initialized: now 1786161272645620 us; error 43 us; skew 500 ppm
I20260808 03:54:32.646049 15962 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:32 dist-test-slave-jlzn krb5kdc[14994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161272, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:32.649966 15962 init.cc:377] Logged in from keytab as kudu/127.26.146.129@KRBTEST.COM (short username kudu)
I20260808 03:54:32.651631 15962 webserver.cc:460] Webserver started at http://127.26.146.129:46613/ using document root <none> and password file <none>
I20260808 03:54:32.652647 15962 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:32.652801 15962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:54:32.659760 15962 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.003s
I20260808 03:54:32.662995 16011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:32.664649 15962 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:54:32.664825 15962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "c90351ca171c4990bbd8672aa972a90a"
format_stamp: "Formatted at 2026-08-08 03:54:29 on dist-test-slave-jlzn"
server_key: "c083362189564de86d89efce478fe81c"
server_key_iv: "c2b261e63244aab02fef66e99512a175"
server_key_version: "encryptionkey@0"
I20260808 03:54:32.666122 15962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:32.684885 15962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:32.688844 15962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:32.689073 15962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:32.689980 15962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:54:32.691251 15962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:54:32.691358 15962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:32.691437 15962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:54:32.691506 15962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:32.719758 15962 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:44607
I20260808 03:54:32.719877 16181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:44607 every 8 connection(s)
I20260808 03:54:32.721222 15962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:54:32.726670 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 15962
I20260808 03:54:32.727289 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:40337
--local_ip_for_outbound_sockets=127.26.146.130
--tserver_master_addrs=127.26.146.190:38815
--webserver_port=38259
--webserver_interface=127.26.146.130
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 03:54:32 dist-test-slave-jlzn krb5kdc[14994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161272, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:54:32.745010 16186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:32.724376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.129:37207)
0808 03:54:32.725197 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:32.725249 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:32.726951 (+ 1702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:32.728416 (+ 1465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:32.728427 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:32.729170 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:32.729791 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:32.729810 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:32.731446 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:32.731450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:32.732721 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:32.732735 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:32.733754 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:32.733762 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:32.734011 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:32.734646 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:32.734669 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:32.736989 (+ 2320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:32.740631 (+ 3642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:32.740640 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:32.740655 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:32.741164 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:32.741600 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:32.741603 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:32.741605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:32.741792 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:32.742534 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:32.742543 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:32.742877 (+ 334us) client_negotiation.cc:769] Sending connection context
0808 03:54:32.743114 (+ 237us) client_negotiation.cc:240] Negotiation successful
0808 03:54:32.743661 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":329,"threads_started":1}
I20260808 03:54:32.745007 16185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:32.724185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:37207 (local address 127.26.146.190:38815)
0808 03:54:32.724894 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:32.724910 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:32.725378 (+ 468us) server_negotiation.cc:407] Connection header received
0808 03:54:32.727515 (+ 2137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:32.727532 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:32.727911 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:32.728240 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:32.730049 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:32.731280 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:32.732864 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:32.733518 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:32.733708 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:32.737268 (+ 3560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:32.737276 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:32.737290 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:32.737331 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:32.740486 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:32.741303 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:32.741310 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:32.741316 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:32.741480 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:32.741913 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:32.741917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:32.741919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:32.742380 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:32.742618 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:32.743372 (+ 754us) server_negotiation.cc:299] Negotiation successful
0808 03:54:32.743759 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":261,"threads_started":1}
I20260808 03:54:32.747745 16182 heartbeater.cc:347] Connected to a master server at 127.26.146.190:38815
I20260808 03:54:32.748072 16182 heartbeater.cc:464] Registering TS with master...
I20260808 03:54:32.748782 16182 heartbeater.cc:511] Master 127.26.146.190:38815 requested a full tablet report, sending...
I20260808 03:54:32.750829 15898 ts_manager.cc:194] Registered new tserver with Master: c90351ca171c4990bbd8672aa972a90a (127.26.146.129:44607)
I20260808 03:54:32.752350 15898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.129@KRBTEST.COM'} at 127.26.146.129:37207
W20260808 03:54:33.056653 16187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:33.056952 16187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:33.057098 16187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:33.064153 16187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:33.064267 16187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:33.064416 16187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:54:33.074055 16187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:40337
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=38259
--tserver_master_addrs=127.26.146.190:38815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:33.075296 16187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:33.076609 16187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:33.089407 16199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:33.089483 16196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:33.089483 16197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:33.090286 16187 server_base.cc:1034] running on GCE node
I20260808 03:54:33.090917 16187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:33.091881 16187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:33.093094 16187 hybrid_clock.cc:648] HybridClock initialized: now 1786161273093039 us; error 44 us; skew 500 ppm
I20260808 03:54:33.093411 16187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:33 dist-test-slave-jlzn krb5kdc[14994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161273, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:33.096614 16187 init.cc:377] Logged in from keytab as kudu/127.26.146.130@KRBTEST.COM (short username kudu)
I20260808 03:54:33.098045 16187 webserver.cc:460] Webserver started at http://127.26.146.130:38259/ using document root <none> and password file <none>
I20260808 03:54:33.098738 16187 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:33.098872 16187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:54:33.104213 16187 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:54:33.106791 16216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:33.108090 16187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:54:33.108244 16187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "303151904f2a433da55e682e01bf35a8"
format_stamp: "Formatted at 2026-08-08 03:54:30 on dist-test-slave-jlzn"
server_key: "4dc4947639b416a64714e5ea60c6aacd"
server_key_iv: "a49cb8d946a3dfdabd5b55cbc3e4d388"
server_key_version: "encryptionkey@0"
I20260808 03:54:33.109218 16187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:33.132578 16187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:33.136737 16187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:33.136986 16187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:33.137954 16187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:54:33.139281 16187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:54:33.139391 16187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:33.139496 16187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:54:33.139567 16187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:33.170217 16187 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:40337
I20260808 03:54:33.170315 16378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:40337 every 8 connection(s)
I20260808 03:54:33.171663 16187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:54:33.174760 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 16187
I20260808 03:54:33.175532 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:34861
--local_ip_for_outbound_sockets=127.26.146.131
--tserver_master_addrs=127.26.146.190:38815
--webserver_port=35781
--webserver_interface=127.26.146.131
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 03:54:33 dist-test-slave-jlzn krb5kdc[14994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161273, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:54:33.191087 16185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:33.174103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:46477 (local address 127.26.146.190:38815)
0808 03:54:33.174291 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:33.174295 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:33.175528 (+ 1233us) server_negotiation.cc:407] Connection header received
0808 03:54:33.177467 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:33.177473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:33.177563 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:33.177649 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:33.179292 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.180059 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.181589 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.182006 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.182118 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:33.185460 (+ 3342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:33.185466 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:33.185469 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:33.185502 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:33.188008 (+ 2506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:33.188937 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:33.188941 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:33.188943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:33.189069 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:33.189518 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:33.189521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:33.189523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:33.189775 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:33.189883 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:33.190656 (+ 773us) server_negotiation.cc:299] Negotiation successful
0808 03:54:33.190830 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 03:54:33.192492 16383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:33.174634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.130:46477)
0808 03:54:33.175372 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:33.175415 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:33.177206 (+ 1791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:33.177900 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:33.177908 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:33.178565 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:33.179123 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:33.179141 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.180208 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.180212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:33.181424 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:33.181435 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.182231 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.182239 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:33.182456 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:33.183105 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:33.183128 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:33.185253 (+ 2125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:33.188149 (+ 2896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:33.188157 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:33.188172 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:33.188798 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:33.189163 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:33.189167 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:33.189170 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:33.189408 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:33.189899 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:33.189906 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:33.190302 (+ 396us) client_negotiation.cc:769] Sending connection context
0808 03:54:33.190592 (+ 290us) client_negotiation.cc:240] Negotiation successful
0808 03:54:33.191097 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":299,"threads_started":1}
I20260808 03:54:33.193969 16380 heartbeater.cc:347] Connected to a master server at 127.26.146.190:38815
I20260808 03:54:33.194339 16380 heartbeater.cc:464] Registering TS with master...
I20260808 03:54:33.195179 16380 heartbeater.cc:511] Master 127.26.146.190:38815 requested a full tablet report, sending...
I20260808 03:54:33.196727 15898 ts_manager.cc:194] Registered new tserver with Master: 303151904f2a433da55e682e01bf35a8 (127.26.146.130:40337)
I20260808 03:54:33.197904 15898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.130@KRBTEST.COM'} at 127.26.146.130:46477
W20260808 03:54:33.505049 16384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:33.505332 16384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:33.505425 16384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:33.512487 16384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:33.512600 16384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:33.512722 16384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:54:33.522212 16384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:34861
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=35781
--tserver_master_addrs=127.26.146.190:38815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:33.523422 16384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:33.524701 16384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:33.537050 16391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:33.537223 16390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:33.537403 16395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:33.537827 16384 server_base.cc:1034] running on GCE node
I20260808 03:54:33.538475 16384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:33.539444 16384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:33.540655 16384 hybrid_clock.cc:648] HybridClock initialized: now 1786161273540604 us; error 51 us; skew 500 ppm
I20260808 03:54:33.540922 16384 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:33 dist-test-slave-jlzn krb5kdc[14994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161273, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:33.543897 16384 init.cc:377] Logged in from keytab as kudu/127.26.146.131@KRBTEST.COM (short username kudu)
I20260808 03:54:33.545204 16384 webserver.cc:460] Webserver started at http://127.26.146.131:35781/ using document root <none> and password file <none>
I20260808 03:54:33.545853 16384 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:33.545984 16384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:54:33.551095 16384 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 03:54:33.553685 16405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:33.554898 16384 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:54:33.555060 16384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "65b52d7e7d524c3983380747aaefc15c"
format_stamp: "Formatted at 2026-08-08 03:54:30 on dist-test-slave-jlzn"
server_key: "e1a1611eff7a5b97e6a36ba64618f124"
server_key_iv: "62ab018894e8bb06ad5f2a4fe520c636"
server_key_version: "encryptionkey@0"
I20260808 03:54:33.556008 16384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:33.568488 16384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:33.572564 16384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:33.572808 16384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:33.573698 16384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:54:33.574961 16384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:54:33.575058 16384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:33.575163 16384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:54:33.575237 16384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:33.605289 16384 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:34861
I20260808 03:54:33.605358 16591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:34861 every 8 connection(s)
I20260808 03:54:33.606817 16384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:54:33.614913 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 16384
Aug 08 03:54:33 dist-test-slave-jlzn krb5kdc[14994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161273, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:54:33.627339 16185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:33.609224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:59429 (local address 127.26.146.190:38815)
0808 03:54:33.609393 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:33.609397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:33.610503 (+ 1106us) server_negotiation.cc:407] Connection header received
0808 03:54:33.612300 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:33.612305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:33.612393 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:33.612492 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:33.613978 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.614821 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.616831 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.617301 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.617413 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:33.621351 (+ 3938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:33.621355 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:33.621359 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:33.621397 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:33.623961 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:33.624826 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:33.624831 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:33.624835 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:33.624992 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:33.625464 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:33.625469 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:33.625473 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:33.625859 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:33.625991 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:33.626696 (+ 705us) server_negotiation.cc:299] Negotiation successful
0808 03:54:33.626989 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 03:54:33.628140 16594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:33.609693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.131:59429)
0808 03:54:33.610362 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:33.610400 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:33.612051 (+ 1651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:33.612746 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:33.612753 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:33.613231 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:33.613819 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:33.613837 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.614985 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.614991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:33.616675 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:33.616689 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.617520 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.617527 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:33.617752 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:33.618457 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:33.618490 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:33.621158 (+ 2668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:33.624201 (+ 3043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:33.624210 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:33.624225 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:33.624700 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:33.625092 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:33.625096 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:33.625100 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:33.625333 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:33.625989 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:33.625997 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:33.626333 (+ 336us) client_negotiation.cc:769] Sending connection context
0808 03:54:33.626555 (+ 222us) client_negotiation.cc:240] Negotiation successful
0808 03:54:33.626927 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":258,"threads_started":1}
I20260808 03:54:33.629489 16592 heartbeater.cc:347] Connected to a master server at 127.26.146.190:38815
I20260808 03:54:33.629773 16592 heartbeater.cc:464] Registering TS with master...
I20260808 03:54:33.630307 16596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:33.615788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60734 (local address 127.26.146.190:38815)
0808 03:54:33.616249 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:33.616255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:33.616547 (+ 292us) server_negotiation.cc:407] Connection header received
0808 03:54:33.616753 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:33.616758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:33.616813 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:33.616904 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:33.618331 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.619427 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.621026 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.621602 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.621734 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:33.624434 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:33.624438 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:33.624441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:33.624468 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:33.627461 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:33.628391 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:33.628395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:33.628397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:33.628507 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:33.628916 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:33.628921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:33.628924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:33.629175 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:33.629368 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:33.629772 (+ 404us) server_negotiation.cc:299] Negotiation successful
0808 03:54:33.629960 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":214,"threads_started":1}
I20260808 03:54:33.630577 16592 heartbeater.cc:511] Master 127.26.146.190:38815 requested a full tablet report, sending...
I20260808 03:54:33.632145 15892 ts_manager.cc:194] Registered new tserver with Master: 65b52d7e7d524c3983380747aaefc15c (127.26.146.131:34861)
I20260808 03:54:33.633136 15892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.131@KRBTEST.COM'} at 127.26.146.131:59429
I20260808 03:54:33.644793 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:54:33.650847 16596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:33.645706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60736 (local address 127.26.146.190:38815)
0808 03:54:33.645853 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:33.645857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:33.646194 (+ 337us) server_negotiation.cc:407] Connection header received
0808 03:54:33.646382 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:33.646386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:33.646512 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:33.646632 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:54:33.646995 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.647658 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.649081 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.649499 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.649589 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:33.649972 (+ 383us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:54:33.650460 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:54:33.650568 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:33.650604 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 03:54:33.650663 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 03:54:33.651744 15892 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:60736:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:54:33.658749 15892 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:54:33.682155 16605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:33.670623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:34861 (local address 127.0.0.1:51270)
0808 03:54:33.671866 (+ 1243us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:33.671884 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:33.672000 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:33.673278 (+ 1278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:33.673286 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:33.673318 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:54:33.673785 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:33.673798 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.676266 (+ 2468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.676272 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:33.678809 (+ 2537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:33.678827 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.680886 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.680894 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:33.681127 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:33.681188 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 03:54:33.681302 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 03:54:33.681761 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1076,"mutex_wait_us":17,"thread_start_us":900,"threads_started":1}
I20260808 03:54:33.682848 16608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:33.670731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51270 (local address 127.26.146.131:34861)
0808 03:54:33.672609 (+ 1878us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:33.672619 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:33.672655 (+ 36us) server_negotiation.cc:407] Connection header received
0808 03:54:33.672760 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:33.672769 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:33.673018 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:33.674029 (+ 1011us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:54:33.674347 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.676097 (+ 1750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.679065 (+ 2968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.680616 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.682083 (+ 1467us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:33.682265 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:33.682348 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 03:54:33.682503 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1700,"thread_start_us":1522,"threads_started":1}
I20260808 03:54:33.687058 16487 tablet_service.cc:1467] Processing CreateTablet for tablet 426fd9bfe482454b8b15d857d8b68597 (DEFAULT_TABLE table=test-table [id=846cfa2487304e76b55cbeeed69e77b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:54:33.689087 16487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 426fd9bfe482454b8b15d857d8b68597. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:54:33.689919 16603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:33.669845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:40337 (local address 127.0.0.1:33958)
0808 03:54:33.670946 (+ 1101us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:33.670986 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:33.671155 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:33.683742 (+ 12587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:33.683747 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:33.683772 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:54:33.684150 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:33.684164 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.685704 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.685709 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:33.687576 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:33.687592 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.689346 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.689350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:33.689477 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:33.689501 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 03:54:33.689563 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 03:54:33.689653 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"mutex_wait_us":23,"thread_start_us":436,"threads_started":1}
I20260808 03:54:33.690804 16604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:33.670070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33958 (local address 127.26.146.130:40337)
0808 03:54:33.683271 (+ 13201us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:33.683279 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:33.683312 (+ 33us) server_negotiation.cc:407] Connection header received
0808 03:54:33.683405 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:33.683411 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:33.683563 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:33.683729 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:54:33.684325 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.685568 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.687753 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.689136 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.690197 (+ 1061us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:33.690351 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:33.690412 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 03:54:33.690520 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13037,"thread_start_us":12908,"threads_started":1}
I20260808 03:54:33.691699 16610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:33.671105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:44607 (local address 127.0.0.1:52310)
0808 03:54:33.680403 (+ 9298us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:33.680424 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:33.680545 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:33.683293 (+ 2748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:33.683298 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:33.683325 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:54:33.683639 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:33.683650 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.686382 (+ 2732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.686387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:33.688837 (+ 2450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:33.688852 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.690815 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.690822 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:33.691093 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:33.691118 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 03:54:33.691189 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 03:54:33.691308 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9161,"mutex_wait_us":1,"thread_start_us":9515,"threads_started":1}
I20260808 03:54:33.691744 16611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:33.671317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52310 (local address 127.26.146.129:44607)
0808 03:54:33.682021 (+ 10704us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:33.682031 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:33.682064 (+ 33us) server_negotiation.cc:407] Connection header received
0808 03:54:33.682164 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:33.682173 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:33.682452 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:33.682651 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:54:33.684096 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.686165 (+ 2069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.689026 (+ 2861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.690590 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.690776 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:33.690931 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:33.691282 (+ 351us) server_negotiation.cc:299] Negotiation successful
0808 03:54:33.691450 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10520,"thread_start_us":10405,"threads_started":1}
I20260808 03:54:33.694763 16285 tablet_service.cc:1467] Processing CreateTablet for tablet 426fd9bfe482454b8b15d857d8b68597 (DEFAULT_TABLE table=test-table [id=846cfa2487304e76b55cbeeed69e77b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:54:33.695081 16085 tablet_service.cc:1467] Processing CreateTablet for tablet 426fd9bfe482454b8b15d857d8b68597 (DEFAULT_TABLE table=test-table [id=846cfa2487304e76b55cbeeed69e77b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:54:33.696194 16285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 426fd9bfe482454b8b15d857d8b68597. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:54:33.696236 16085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 426fd9bfe482454b8b15d857d8b68597. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:54:33.706441 16613 tablet_bootstrap.cc:492] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c: Bootstrap starting.
I20260808 03:54:33.709239 16614 tablet_bootstrap.cc:492] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a: Bootstrap starting.
I20260808 03:54:33.710165 16613 tablet_bootstrap.cc:654] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c: Neither blocks nor log segments found. Creating new log.
I20260808 03:54:33.711169 16613 log.cc:824] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c: Log is configured to *not* fsync() on all Append() calls
I20260808 03:54:33.713124 16614 tablet_bootstrap.cc:654] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a: Neither blocks nor log segments found. Creating new log.
I20260808 03:54:33.714083 16613 tablet_bootstrap.cc:492] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c: No bootstrap required, opened a new log
I20260808 03:54:33.714185 16614 log.cc:824] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a: Log is configured to *not* fsync() on all Append() calls
I20260808 03:54:33.714331 16613 ts_tablet_manager.cc:1397] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 03:54:33.715517 16615 tablet_bootstrap.cc:492] T 426fd9bfe482454b8b15d857d8b68597 P 303151904f2a433da55e682e01bf35a8: Bootstrap starting.
I20260808 03:54:33.717236 16614 tablet_bootstrap.cc:492] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a: No bootstrap required, opened a new log
I20260808 03:54:33.717511 16614 ts_tablet_manager.cc:1397] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 03:54:33.719136 16615 tablet_bootstrap.cc:654] T 426fd9bfe482454b8b15d857d8b68597 P 303151904f2a433da55e682e01bf35a8: Neither blocks nor log segments found. Creating new log.
I20260808 03:54:33.720433 16615 log.cc:824] T 426fd9bfe482454b8b15d857d8b68597 P 303151904f2a433da55e682e01bf35a8: Log is configured to *not* fsync() on all Append() calls
I20260808 03:54:33.722489 16613 raft_consensus.cc:348] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } }
I20260808 03:54:33.722832 16613 raft_consensus.cc:374] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:54:33.722985 16613 raft_consensus.cc:729] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65b52d7e7d524c3983380747aaefc15c, State: Initialized, Role: FOLLOWER
I20260808 03:54:33.723594 16615 tablet_bootstrap.cc:492] T 426fd9bfe482454b8b15d857d8b68597 P 303151904f2a433da55e682e01bf35a8: No bootstrap required, opened a new log
I20260808 03:54:33.724018 16613 consensus_queue.cc:260] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } }
I20260808 03:54:33.724435 16615 ts_tablet_manager.cc:1397] T 426fd9bfe482454b8b15d857d8b68597 P 303151904f2a433da55e682e01bf35a8: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.005s
I20260808 03:54:33.730047 16614 raft_consensus.cc:348] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } }
I20260808 03:54:33.730557 16614 raft_consensus.cc:374] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:54:33.730696 16614 raft_consensus.cc:729] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c90351ca171c4990bbd8672aa972a90a, State: Initialized, Role: FOLLOWER
I20260808 03:54:33.731452 16614 consensus_queue.cc:260] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } }
I20260808 03:54:33.736456 16615 raft_consensus.cc:348] T 426fd9bfe482454b8b15d857d8b68597 P 303151904f2a433da55e682e01bf35a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } }
I20260808 03:54:33.737429 16615 raft_consensus.cc:374] T 426fd9bfe482454b8b15d857d8b68597 P 303151904f2a433da55e682e01bf35a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:54:33.737543 16592 heartbeater.cc:503] Master 127.26.146.190:38815 was elected leader, sending a full tablet report...
I20260808 03:54:33.737598 16615 raft_consensus.cc:729] T 426fd9bfe482454b8b15d857d8b68597 P 303151904f2a433da55e682e01bf35a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 303151904f2a433da55e682e01bf35a8, State: Initialized, Role: FOLLOWER
I20260808 03:54:33.738412 16613 ts_tablet_manager.cc:1428] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c: Time spent starting tablet: real 0.024s user 0.011s sys 0.013s
I20260808 03:54:33.738279 16615 consensus_queue.cc:260] T 426fd9bfe482454b8b15d857d8b68597 P 303151904f2a433da55e682e01bf35a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } }
I20260808 03:54:33.739576 16182 heartbeater.cc:503] Master 127.26.146.190:38815 was elected leader, sending a full tablet report...
I20260808 03:54:33.740242 16614 ts_tablet_manager.cc:1428] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a: Time spent starting tablet: real 0.023s user 0.022s sys 0.000s
I20260808 03:54:33.741263 16380 heartbeater.cc:503] Master 127.26.146.190:38815 was elected leader, sending a full tablet report...
I20260808 03:54:33.741789 16615 ts_tablet_manager.cc:1428] T 426fd9bfe482454b8b15d857d8b68597 P 303151904f2a433da55e682e01bf35a8: Time spent starting tablet: real 0.017s user 0.009s sys 0.004s
W20260808 03:54:33.858434 16593 tablet.cc:2369] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:54:33.923558 16381 tablet.cc:2369] T 426fd9bfe482454b8b15d857d8b68597 P 303151904f2a433da55e682e01bf35a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:54:33.973519 16184 tablet.cc:2369] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:54:33.979521 16621 raft_consensus.cc:482] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:54:33.979828 16621 raft_consensus.cc:504] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } }
I20260808 03:54:33.981555 16621 leader_election.cc:288] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 303151904f2a433da55e682e01bf35a8 (127.26.146.130:40337), c90351ca171c4990bbd8672aa972a90a (127.26.146.129:44607)
I20260808 03:54:33.987784 16604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:33.981956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:46439 (local address 127.26.146.130:40337)
0808 03:54:33.982170 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:33.982176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:33.982261 (+ 85us) server_negotiation.cc:407] Connection header received
0808 03:54:33.982519 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:33.982525 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:33.982618 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:33.982713 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:54:33.983126 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.984094 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.985851 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.986918 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.987037 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:33.987093 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:33.987410 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 03:54:33.987497 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 03:54:33.987833 16594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:33.981746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:40337 (local address 127.26.146.131:46439)
0808 03:54:33.982169 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:33.982188 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:33.982320 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:33.982748 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:33.982752 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:33.982778 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:54:33.982976 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:33.982987 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.984228 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.984234 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:33.985696 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:33.985709 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.987063 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.987069 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:33.987264 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:33.987299 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 03:54:33.987396 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 03:54:33.987519 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":1}
I20260808 03:54:33.988473 16611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:33.982220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:56229 (local address 127.26.146.129:44607)
0808 03:54:33.982415 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:33.982421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:33.982863 (+ 442us) server_negotiation.cc:407] Connection header received
0808 03:54:33.983013 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:33.983018 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:33.983093 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:33.983163 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:54:33.983729 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.984623 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.986561 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.987497 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.987635 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:33.987710 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:33.988116 (+ 406us) server_negotiation.cc:299] Negotiation successful
0808 03:54:33.988238 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 03:54:33.988555 16625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:33.982077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:44607 (local address 127.26.146.131:56229)
0808 03:54:33.982757 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:33.982774 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:33.982891 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:33.983203 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:33.983207 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:33.983232 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:54:33.983577 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:33.983589 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.984792 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.984796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:33.986401 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:33.986416 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:33.987696 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:33.987702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:33.987898 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:33.987931 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 03:54:33.988072 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 03:54:33.988194 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"mutex_wait_us":20,"thread_start_us":365,"threads_started":1}
I20260808 03:54:33.988904 16312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "426fd9bfe482454b8b15d857d8b68597" candidate_uuid: "65b52d7e7d524c3983380747aaefc15c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "303151904f2a433da55e682e01bf35a8" is_pre_election: true
I20260808 03:54:33.989370 16312 raft_consensus.cc:2454] T 426fd9bfe482454b8b15d857d8b68597 P 303151904f2a433da55e682e01bf35a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65b52d7e7d524c3983380747aaefc15c in term 0.
I20260808 03:54:33.989472 16115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "426fd9bfe482454b8b15d857d8b68597" candidate_uuid: "65b52d7e7d524c3983380747aaefc15c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c90351ca171c4990bbd8672aa972a90a" is_pre_election: true
I20260808 03:54:33.989992 16417 leader_election.cc:302] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 303151904f2a433da55e682e01bf35a8, 65b52d7e7d524c3983380747aaefc15c; no voters:
I20260808 03:54:33.990024 16115 raft_consensus.cc:2454] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65b52d7e7d524c3983380747aaefc15c in term 0.
I20260808 03:54:33.990422 16621 raft_consensus.cc:2781] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:54:33.990592 16621 raft_consensus.cc:482] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:54:33.990692 16621 raft_consensus.cc:3037] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:54:33.992899 16621 raft_consensus.cc:504] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } }
I20260808 03:54:33.993469 16621 leader_election.cc:288] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c [CANDIDATE]: Term 1 election: Requested vote from peers 303151904f2a433da55e682e01bf35a8 (127.26.146.130:40337), c90351ca171c4990bbd8672aa972a90a (127.26.146.129:44607)
I20260808 03:54:33.993934 16312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "426fd9bfe482454b8b15d857d8b68597" candidate_uuid: "65b52d7e7d524c3983380747aaefc15c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "303151904f2a433da55e682e01bf35a8"
I20260808 03:54:33.993963 16115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "426fd9bfe482454b8b15d857d8b68597" candidate_uuid: "65b52d7e7d524c3983380747aaefc15c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c90351ca171c4990bbd8672aa972a90a"
I20260808 03:54:33.994151 16115 raft_consensus.cc:3037] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:54:33.994156 16312 raft_consensus.cc:3037] T 426fd9bfe482454b8b15d857d8b68597 P 303151904f2a433da55e682e01bf35a8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:54:33.997296 16312 raft_consensus.cc:2454] T 426fd9bfe482454b8b15d857d8b68597 P 303151904f2a433da55e682e01bf35a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65b52d7e7d524c3983380747aaefc15c in term 1.
I20260808 03:54:33.997296 16115 raft_consensus.cc:2454] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65b52d7e7d524c3983380747aaefc15c in term 1.
I20260808 03:54:33.997648 16417 leader_election.cc:302] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65b52d7e7d524c3983380747aaefc15c, c90351ca171c4990bbd8672aa972a90a; no voters:
I20260808 03:54:33.997900 16621 raft_consensus.cc:2781] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:54:33.998323 16621 raft_consensus.cc:686] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c [term 1 LEADER]: Becoming Leader. State: Replica: 65b52d7e7d524c3983380747aaefc15c, State: Running, Role: LEADER
I20260808 03:54:33.998768 16621 consensus_queue.cc:237] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } }
I20260808 03:54:34.003629 15891 catalog_manager.cc:5495] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c reported cstate change: term changed from 0 to 1, leader changed from <none> to 65b52d7e7d524c3983380747aaefc15c (127.26.146.131). New cstate: current_term: 1 leader_uuid: "65b52d7e7d524c3983380747aaefc15c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } health_report { overall_health: HEALTHY } } }
I20260808 03:54:34.059475 16608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:34.054682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51280 (local address 127.26.146.131:34861)
0808 03:54:34.054844 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:34.054849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:34.054866 (+ 17us) server_negotiation.cc:407] Connection header received
0808 03:54:34.055043 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:34.055053 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:34.055130 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:34.055252 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:54:34.055532 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:34.056278 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:34.057591 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:34.057977 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:34.058091 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:34.058477 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:54:34.059025 (+ 548us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:54:34.059155 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:34.059222 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 03:54:34.059282 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 03:54:34.067119 16312 raft_consensus.cc:1261] T 426fd9bfe482454b8b15d857d8b68597 P 303151904f2a433da55e682e01bf35a8 [term 1 FOLLOWER]: Refusing update from remote peer 65b52d7e7d524c3983380747aaefc15c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:54:34.067075 16115 raft_consensus.cc:1261] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Refusing update from remote peer 65b52d7e7d524c3983380747aaefc15c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:54:34.068049 16626 consensus_queue.cc:1035] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:54:34.068418 16621 consensus_queue.cc:1035] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c [LEADER]: Connected to new peer: Peer: permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:54:34.079308 16630 mvcc.cc:205] Tried to move back new op lower bound from 7316116578568916992 to 7316116578303057920. Current Snapshot: MvccSnapshot[applied={T|T < 7316116578568916992}]
I20260808 03:54:34.084429 16632 mvcc.cc:205] Tried to move back new op lower bound from 7316116578568916992 to 7316116578303057920. Current Snapshot: MvccSnapshot[applied={T|T < 7316116578568916992}]
I20260808 03:54:34.087612 16631 mvcc.cc:205] Tried to move back new op lower bound from 7316116578568916992 to 7316116578303057920. Current Snapshot: MvccSnapshot[applied={T|T < 7316116578568916992 or (T in {7316116578568916992})}]
I20260808 03:54:34.095991 15891 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:60736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:54:34.096252 15891 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:60736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:54:34.099745 15891 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703: Sending DeleteTablet for 3 replicas of tablet 426fd9bfe482454b8b15d857d8b68597
I20260808 03:54:34.100936 16085 tablet_service.cc:1514] Processing DeleteTablet for tablet 426fd9bfe482454b8b15d857d8b68597 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:54:34 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:52310
I20260808 03:54:34.101006 16487 tablet_service.cc:1514] Processing DeleteTablet for tablet 426fd9bfe482454b8b15d857d8b68597 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:54:34 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:51270
I20260808 03:54:34.101419 16285 tablet_service.cc:1514] Processing DeleteTablet for tablet 426fd9bfe482454b8b15d857d8b68597 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:54:34 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:33958
I20260808 03:54:34.102681 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 15962
I20260808 03:54:34.112082 16647 tablet_replica.cc:330] T 426fd9bfe482454b8b15d857d8b68597 P 303151904f2a433da55e682e01bf35a8: stopping tablet replica
I20260808 03:54:34.113236 16647 raft_consensus.cc:2229] T 426fd9bfe482454b8b15d857d8b68597 P 303151904f2a433da55e682e01bf35a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:54:34.114452 16647 raft_consensus.cc:2258] T 426fd9bfe482454b8b15d857d8b68597 P 303151904f2a433da55e682e01bf35a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:54:34.117801 16647 ts_tablet_manager.cc:1905] T 426fd9bfe482454b8b15d857d8b68597 P 303151904f2a433da55e682e01bf35a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:54:34.122706 16611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:34.104351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52314 (local address 127.26.146.129:44607)
0808 03:54:34.104531 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:34.104537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:34.104556 (+ 19us) server_negotiation.cc:407] Connection header received
0808 03:54:34.104615 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:34.104621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:34.104703 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:34.104796 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:34.106854 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:34.108091 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:34.110280 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:34.110952 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:34.111064 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:34.115152 (+ 4088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:34.115170 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:34.115191 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:34.115245 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:34.119125 (+ 3880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:34.120050 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:34.120060 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:34.120064 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:34.120253 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:34.120680 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:34.120685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:34.120689 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:34.121182 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:34.121504 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:34.121866 (+ 362us) server_negotiation.cc:299] Negotiation successful
0808 03:54:34.122114 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 03:54:34.123513 16176 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:34.127489 16647 ts_tablet_manager.cc:1918] T 426fd9bfe482454b8b15d857d8b68597 P 303151904f2a433da55e682e01bf35a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:54:34.127713 16647 log.cc:1196] T 426fd9bfe482454b8b15d857d8b68597 P 303151904f2a433da55e682e01bf35a8: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/426fd9bfe482454b8b15d857d8b68597
I20260808 03:54:34.128324 16647 ts_tablet_manager.cc:1939] T 426fd9bfe482454b8b15d857d8b68597 P 303151904f2a433da55e682e01bf35a8: Deleting consensus metadata
I20260808 03:54:34.129293 16646 tablet_replica.cc:330] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c: stopping tablet replica
I20260808 03:54:34.129726 16646 raft_consensus.cc:2229] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:54:34.130270 15871 catalog_manager.cc:4841] TS 303151904f2a433da55e682e01bf35a8 (127.26.146.130:40337): tablet 426fd9bfe482454b8b15d857d8b68597 (table test-table [id=846cfa2487304e76b55cbeeed69e77b0]) successfully deleted
I20260808 03:54:34.130501 16646 raft_consensus.cc:2258] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:54:34.132691 16646 ts_tablet_manager.cc:1905] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:54:34.140275 16646 ts_tablet_manager.cc:1918] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:54:34.140460 16646 log.cc:1196] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/426fd9bfe482454b8b15d857d8b68597
I20260808 03:54:34.141027 16646 ts_tablet_manager.cc:1939] T 426fd9bfe482454b8b15d857d8b68597 P 65b52d7e7d524c3983380747aaefc15c: Deleting consensus metadata
I20260808 03:54:34.142728 15870 catalog_manager.cc:4841] TS 65b52d7e7d524c3983380747aaefc15c (127.26.146.131:34861): tablet 426fd9bfe482454b8b15d857d8b68597 (table test-table [id=846cfa2487304e76b55cbeeed69e77b0]) successfully deleted
I20260808 03:54:34.233093 16645 tablet_replica.cc:330] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a: stopping tablet replica
I20260808 03:54:34.233605 16645 raft_consensus.cc:2229] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:54:34.234030 16645 raft_consensus.cc:2258] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:54:34.234426 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 15962
I20260808 03:54:34.258852 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 16187
W20260808 03:54:34.258836 15871 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.146.129:44607 (error 108)
I20260808 03:54:34.260918 16650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:34.260241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:44607 (local address 127.0.0.1:52322)
0808 03:54:34.260646 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:34.260762 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:44607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":208,"threads_started":1}
W20260808 03:54:34.261221 15871 catalog_manager.cc:4584] TS c90351ca171c4990bbd8672aa972a90a (127.26.146.129:44607): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 426fd9bfe482454b8b15d857d8b68597: Network error: Client connection negotiation failed: client connection to 127.26.146.129:44607: connect: Connection refused (error 111)
I20260808 03:54:34.277935 16604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:34.259851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33964 (local address 127.26.146.130:40337)
0808 03:54:34.260024 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:34.260029 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:34.260165 (+ 136us) server_negotiation.cc:407] Connection header received
0808 03:54:34.260421 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:34.260428 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:34.260516 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:34.260620 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:34.262472 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:34.263185 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:34.265088 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:34.265496 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:34.265588 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:34.272124 (+ 6536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:34.272140 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:34.272157 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:34.272203 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:34.275056 (+ 2853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:34.275943 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:34.275952 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:34.275955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:34.276119 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:34.276500 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:34.276503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:34.276505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:34.276881 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:34.277127 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:34.277455 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 03:54:34.277632 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 03:54:34.278693 16374 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:34.315218 16650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:34.314618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:44607 (local address 127.0.0.1:52324)
0808 03:54:34.314894 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:34.315023 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:44607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260808 03:54:34.316506 16650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:34.316064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:44607 (local address 127.0.0.1:52334)
0808 03:54:34.316247 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:34.316339 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:44607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 03:54:34.368742 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 16187
I20260808 03:54:34.377388 16650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:34.376820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:44607 (local address 127.0.0.1:52348)
0808 03:54:34.377071 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:34.377193 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:44607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260808 03:54:34.378706 16650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:34.378276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:44607 (local address 127.0.0.1:52352)
0808 03:54:34.378443 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:34.378543 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:44607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 03:54:34.392113 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 16384
I20260808 03:54:34.406144 16608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:34.392853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51282 (local address 127.26.146.131:34861)
0808 03:54:34.393034 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:34.393041 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:34.393171 (+ 130us) server_negotiation.cc:407] Connection header received
0808 03:54:34.393370 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:34.393375 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:34.393443 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:34.393536 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:34.394884 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:34.395748 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:34.397082 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:34.397508 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:34.397658 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:34.400219 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:34.400232 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:34.400250 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:34.400287 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:34.403362 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:34.404072 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:34.404079 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:34.404082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:34.404234 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:34.404614 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:34.404617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:34.404619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:34.404970 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:34.405276 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:34.405578 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 03:54:34.405787 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 03:54:34.406876 16585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:34.459420 16650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:34.458904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:44607 (local address 127.0.0.1:52358)
0808 03:54:34.459147 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:34.459250 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:44607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260808 03:54:34.460608 16650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:34.460155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:44607 (local address 127.0.0.1:52366)
0808 03:54:34.460370 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:34.460447 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:44607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260808 03:54:34.527237 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 16384
I20260808 03:54:34.551622 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 15844
I20260808 03:54:34.552757 15953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:34.665198 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 15844
I20260808 03:54:34.692895 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:38815
--webserver_interface=127.26.146.190
--webserver_port=36151
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:38815
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:54:35.026157 16656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:35.026437 16656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:35.026567 16656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:35.033857 16656 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:54:35.033951 16656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:35.034021 16656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:35.034108 16656 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:54:35.034169 16656 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:54:35.043573 16656 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:38815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:38815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=36151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:35.044791 16656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:35.046016 16656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:35.057168 16664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:35.057271 16665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:35.057302 16667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:35.058456 16656 server_base.cc:1034] running on GCE node
I20260808 03:54:35.059160 16656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:35.060460 16656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:35.061658 16656 hybrid_clock.cc:648] HybridClock initialized: now 1786161275061603 us; error 39 us; skew 500 ppm
I20260808 03:54:35.061957 16656 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:35 dist-test-slave-jlzn krb5kdc[14994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161275, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:35.064985 16656 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:54:35.066344 16656 webserver.cc:460] Webserver started at http://127.26.146.190:36151/ using document root <none> and password file <none>
I20260808 03:54:35.066977 16656 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:35.067112 16656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:54:35.072412 16656 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:54:35.075363 16675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:35.076753 16656 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:54:35.076922 16656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "33d149fcb1654495bdb4ae000beba703"
format_stamp: "Formatted at 2026-08-08 03:54:29 on dist-test-slave-jlzn"
server_key: "93448f31cad4011407b74119899ec8d1"
server_key_iv: "9eafa4580d7aa7d180b9591c3b1ef03d"
server_key_version: "encryptionkey@0"
I20260808 03:54:35.077941 16656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:35.110898 16656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:35.115304 16656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:35.115604 16656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:35.132943 16656 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:38815
I20260808 03:54:35.132964 16749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:38815 every 8 connection(s)
I20260808 03:54:35.134374 16656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:54:35.138888 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 16656
I20260808 03:54:35.139719 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:44607
--local_ip_for_outbound_sockets=127.26.146.129
--tserver_master_addrs=127.26.146.190:38815
--webserver_port=46613
--webserver_interface=127.26.146.129
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:54:35.141202 16751 sys_catalog.cc:263] Verifying existing consensus state
I20260808 03:54:35.144403 16751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703: Bootstrap starting.
I20260808 03:54:35.168849 16751 log.cc:824] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703: Log is configured to *not* fsync() on all Append() calls
I20260808 03:54:35.189955 16751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 03:54:35.190579 16751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703: Bootstrap complete.
I20260808 03:54:35.203716 16751 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d149fcb1654495bdb4ae000beba703" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38815 } }
I20260808 03:54:35.204989 16751 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d149fcb1654495bdb4ae000beba703, State: Initialized, Role: FOLLOWER
I20260808 03:54:35.205710 16751 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d149fcb1654495bdb4ae000beba703" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38815 } }
I20260808 03:54:35.206007 16751 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:54:35.206139 16751 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:54:35.206338 16751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 2 FOLLOWER]: Advancing to term 3
I20260808 03:54:35.211864 16751 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d149fcb1654495bdb4ae000beba703" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38815 } }
I20260808 03:54:35.212527 16751 leader_election.cc:302] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33d149fcb1654495bdb4ae000beba703; no voters:
I20260808 03:54:35.213263 16751 leader_election.cc:288] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 03:54:35.213378 16759 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 03:54:35.214509 16759 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [term 3 LEADER]: Becoming Leader. State: Replica: 33d149fcb1654495bdb4ae000beba703, State: Running, Role: LEADER
I20260808 03:54:35.215047 16759 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d149fcb1654495bdb4ae000beba703" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38815 } }
I20260808 03:54:35.215533 16751 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:54:35.217388 16764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33d149fcb1654495bdb4ae000beba703. Latest consensus state: current_term: 3 leader_uuid: "33d149fcb1654495bdb4ae000beba703" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d149fcb1654495bdb4ae000beba703" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38815 } } }
I20260808 03:54:35.217763 16764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [sys.catalog]: This master's current role is: LEADER
I20260808 03:54:35.218309 16763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "33d149fcb1654495bdb4ae000beba703" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d149fcb1654495bdb4ae000beba703" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38815 } } }
I20260808 03:54:35.218596 16763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33d149fcb1654495bdb4ae000beba703 [sys.catalog]: This master's current role is: LEADER
I20260808 03:54:35.228828 16768 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:54:35.237174 16768 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:54:35.238853 16768 catalog_manager.cc:1259] Loaded cluster ID: ab1ef5b242444f3896007b46d519ed8a
I20260808 03:54:35.239413 16768 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 03:54:35.241632 16768 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fef62cdd9ad google::LogMessage::Fail() at ??:0
@ 0x7fef62ce088a google::LogMessage::SendToLog() at ??:0
@ 0x7fef62cdd4de google::LogMessage::Flush() at ??:0
@ 0x7fef62cdea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fef78710512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fef7870d976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fef78795610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fef78795439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260808 03:54:35.280372 16789 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7fef65929345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fef6592c91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fef6592c799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fef65907eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fef68be16db start_thread at ??:0
@ 0x7fef61b7761f clone at ??:0
W20260808 03:54:35.551760 16753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:35.552062 16753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:35.552183 16753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:35.559333 16753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:35.559441 16753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:35.559597 16753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:54:35.570634 16753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:44607
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=46613
--tserver_master_addrs=127.26.146.190:38815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:35.571979 16753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:35.573308 16753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:35.586160 16815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:35.586216 16817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:35.586239 16820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:35.587281 16753 server_base.cc:1034] running on GCE node
I20260808 03:54:35.587961 16753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:35.588873 16753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:35.590061 16753 hybrid_clock.cc:648] HybridClock initialized: now 1786161275590003 us; error 54 us; skew 500 ppm
I20260808 03:54:35.590354 16753 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:35 dist-test-slave-jlzn krb5kdc[14994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161275, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:35.593429 16753 init.cc:377] Logged in from keytab as kudu/127.26.146.129@KRBTEST.COM (short username kudu)
I20260808 03:54:35.594870 16753 webserver.cc:460] Webserver started at http://127.26.146.129:46613/ using document root <none> and password file <none>
I20260808 03:54:35.595527 16753 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:35.595649 16753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:54:35.600819 16753 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 03:54:35.606129 16828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:35.607354 16753 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 03:54:35.607504 16753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "c90351ca171c4990bbd8672aa972a90a"
format_stamp: "Formatted at 2026-08-08 03:54:29 on dist-test-slave-jlzn"
server_key: "c083362189564de86d89efce478fe81c"
server_key_iv: "c2b261e63244aab02fef66e99512a175"
server_key_version: "encryptionkey@0"
I20260808 03:54:35.608501 16753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:35.624241 16753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:35.628017 16753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:35.628258 16753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:35.629190 16753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:54:35.630918 16842 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260808 03:54:35.636478 16753 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 03:54:35.636587 16753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.001s
I20260808 03:54:35.636705 16753 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 03:54:35.639911 16753 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 03:54:35.639990 16753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260808 03:54:35.640311 16842 tablet_bootstrap.cc:492] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a: Bootstrap starting.
I20260808 03:54:35.665599 16842 log.cc:824] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a: Log is configured to *not* fsync() on all Append() calls
I20260808 03:54:35.678594 16842 tablet_bootstrap.cc:492] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 03:54:35.679235 16842 tablet_bootstrap.cc:492] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a: Bootstrap complete.
I20260808 03:54:35.680680 16842 ts_tablet_manager.cc:1397] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a: Time spent bootstrapping tablet: real 0.041s user 0.020s sys 0.019s
I20260808 03:54:35.681113 16753 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:44607
I20260808 03:54:35.681185 17025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:44607 every 8 connection(s)
I20260808 03:54:35.682363 16753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:54:35.684482 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 16753
I20260808 03:54:35.685096 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:40337
--local_ip_for_outbound_sockets=127.26.146.130
--tserver_master_addrs=127.26.146.190:38815
--webserver_port=38259
--webserver_interface=127.26.146.130
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:54:35.687997 16842 raft_consensus.cc:348] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } }
I20260808 03:54:35.691160 16842 raft_consensus.cc:729] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c90351ca171c4990bbd8672aa972a90a, State: Initialized, Role: FOLLOWER
I20260808 03:54:35.691774 17032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:35.689323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.129:40043)
0808 03:54:35.690145 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:35.690439 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":358,"threads_started":1}
I20260808 03:54:35.691870 16842 consensus_queue.cc:260] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } }
W20260808 03:54:35.692602 17030 heartbeater.cc:644] Failed to heartbeat to 127.26.146.190:38815 (0 consecutive failures): Network error: Failed to ping master at 127.26.146.190:38815: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
I20260808 03:54:35.693593 17032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:35.693178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.129:37213)
0808 03:54:35.693348 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:35.693444 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 03:54:35.694605 17032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:35.694199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.129:48383)
0808 03:54:35.694341 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:35.694440 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 03:54:35.694905 17030 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 03:54:35.696890 17032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:35.696576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.129:59975)
0808 03:54:35.696723 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:35.696786 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 03:54:35.697120 16842 ts_tablet_manager.cc:1428] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
I20260808 03:54:35.698010 17032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:35.697709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.129:58301)
0808 03:54:35.697845 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:35.697905 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 03:54:36.020710 17034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:36.020998 17034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:36.021126 17034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:36.028191 17034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:36.028317 17034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:36.028474 17034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:54:36.038414 17034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:40337
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=38259
--tserver_master_addrs=127.26.146.190:38815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:36.039778 17034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:36.041118 17034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:36.053804 17043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:36.053861 17042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:36.054035 17046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:36.054518 17034 server_base.cc:1034] running on GCE node
I20260808 03:54:36.055105 17034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:36.056012 17034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:36.057202 17034 hybrid_clock.cc:648] HybridClock initialized: now 1786161276057151 us; error 43 us; skew 500 ppm
I20260808 03:54:36.057492 17034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:36 dist-test-slave-jlzn krb5kdc[14994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161276, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:36.060518 17034 init.cc:377] Logged in from keytab as kudu/127.26.146.130@KRBTEST.COM (short username kudu)
I20260808 03:54:36.061872 17034 webserver.cc:460] Webserver started at http://127.26.146.130:38259/ using document root <none> and password file <none>
I20260808 03:54:36.062529 17034 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:36.062638 17034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:54:36.067868 17034 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 03:54:36.070461 17059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:36.071825 17034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 03:54:36.071988 17034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "303151904f2a433da55e682e01bf35a8"
format_stamp: "Formatted at 2026-08-08 03:54:30 on dist-test-slave-jlzn"
server_key: "4dc4947639b416a64714e5ea60c6aacd"
server_key_iv: "a49cb8d946a3dfdabd5b55cbc3e4d388"
server_key_version: "encryptionkey@0"
I20260808 03:54:36.072954 17034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:36.089462 17034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:36.093456 17034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:36.093711 17034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:36.094653 17034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:54:36.096028 17034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:54:36.096134 17034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:36.096241 17034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:54:36.096311 17034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:36.127223 17034 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:40337
I20260808 03:54:36.127280 17254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:40337 every 8 connection(s)
I20260808 03:54:36.128732 17034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:54:36.133529 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 17034
I20260808 03:54:36.133782 17260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:36.131751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.130:59643)
0808 03:54:36.132466 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:36.132678 (+ 212us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":261,"threads_started":1}
I20260808 03:54:36.134281 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:34861
--local_ip_for_outbound_sockets=127.26.146.131
--tserver_master_addrs=127.26.146.190:38815
--webserver_port=35781
--webserver_interface=127.26.146.131
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:54:36.134478 17255 heartbeater.cc:644] Failed to heartbeat to 127.26.146.190:38815 (0 consecutive failures): Network error: Failed to ping master at 127.26.146.190:38815: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
I20260808 03:54:36.135419 17260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:36.135015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.130:58989)
0808 03:54:36.135194 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:36.135282 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 03:54:36.136350 17260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:36.135982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.130:51431)
0808 03:54:36.136149 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:36.136233 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260808 03:54:36.136600 17255 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 03:54:36.468559 17261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:36.468866 17261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:36.469003 17261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:36.476370 17261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:36.476500 17261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:36.476662 17261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:54:36.486289 17261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:34861
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=35781
--tserver_master_addrs=127.26.146.190:38815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:36.487622 17261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:36.488981 17261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:36.502516 17269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:36.502439 17271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:36.502476 17275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:36.508731 17261 server_base.cc:1034] running on GCE node
I20260808 03:54:36.509359 17261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:36.510363 17261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:36.511658 17261 hybrid_clock.cc:648] HybridClock initialized: now 1786161276511588 us; error 75 us; skew 500 ppm
I20260808 03:54:36.512007 17261 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:36 dist-test-slave-jlzn krb5kdc[14994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161276, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:36.515225 17261 init.cc:377] Logged in from keytab as kudu/127.26.146.131@KRBTEST.COM (short username kudu)
I20260808 03:54:36.516768 17261 webserver.cc:460] Webserver started at http://127.26.146.131:35781/ using document root <none> and password file <none>
I20260808 03:54:36.517477 17261 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:36.517604 17261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:54:36.523085 17261 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 03:54:36.525880 17289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:36.527230 17261 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 03:54:36.527401 17261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "65b52d7e7d524c3983380747aaefc15c"
format_stamp: "Formatted at 2026-08-08 03:54:30 on dist-test-slave-jlzn"
server_key: "e1a1611eff7a5b97e6a36ba64618f124"
server_key_iv: "62ab018894e8bb06ad5f2a4fe520c636"
server_key_version: "encryptionkey@0"
I20260808 03:54:36.528453 17261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:36.544821 17261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:36.549659 17261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:36.549928 17261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:36.550892 17261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:54:36.552311 17261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:54:36.552408 17261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:36.552515 17261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:54:36.552587 17261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:36.584352 17261 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:34861
I20260808 03:54:36.584421 17457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:34861 every 8 connection(s)
I20260808 03:54:36.585801 17261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:54:36.590677 17460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:36.588749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.131:60947)
0808 03:54:36.589509 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:36.589762 (+ 253us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":291,"threads_started":1}
W20260808 03:54:36.591288 17458 heartbeater.cc:644] Failed to heartbeat to 127.26.146.190:38815 (0 consecutive failures): Network error: Failed to ping master at 127.26.146.190:38815: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
I20260808 03:54:36.592319 17460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:36.591828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.131:34673)
0808 03:54:36.592056 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:36.592155 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 03:54:36.592634 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 17261
I20260808 03:54:36.593282 17460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:36.592866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.131:42081)
0808 03:54:36.593034 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:36.593136 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 03:54:36.593521 17458 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 03:54:36.699546 17462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:36.698798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.129:56405)
0808 03:54:36.699263 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:36.699344 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":208,"threads_started":1}
I20260808 03:54:37.137914 17463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:37.137087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.130:33999)
0808 03:54:37.137523 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:37.137653 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":214,"threads_started":1}
I20260808 03:54:37.313342 17465 raft_consensus.cc:482] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:54:37.313593 17465 raft_consensus.cc:504] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } }
I20260808 03:54:37.315084 17465 leader_election.cc:288] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 303151904f2a433da55e682e01bf35a8 (127.26.146.130:40337), 65b52d7e7d524c3983380747aaefc15c (127.26.146.131:34861)
Aug 08 03:54:37 dist-test-slave-jlzn krb5kdc[14994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161275, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.130@KRBTEST.COM
Aug 08 03:54:37 dist-test-slave-jlzn krb5kdc[14994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161275, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.131@KRBTEST.COM
I20260808 03:54:37.347648 17472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:37.315771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:55793 (local address 127.26.146.130:40337)
0808 03:54:37.328989 (+ 13218us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:37.329004 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:37.329052 (+ 48us) server_negotiation.cc:407] Connection header received
0808 03:54:37.330542 (+ 1490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:37.330551 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:37.330860 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:37.331128 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:37.333286 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:37.334597 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:37.336375 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:37.336984 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:37.337148 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:37.341109 (+ 3961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:37.341116 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:37.341130 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:37.341169 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:37.343902 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:37.344770 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:37.344780 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:37.344787 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:37.344958 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:37.345375 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:37.345381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:37.345385 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:37.345931 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:37.346112 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:37.346872 (+ 760us) server_negotiation.cc:299] Negotiation successful
0808 03:54:37.347261 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13059,"thread_start_us":12893,"threads_started":1}
I20260808 03:54:37.347663 17470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:37.315359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:40337 (local address 127.26.146.129:55793)
0808 03:54:37.327995 (+ 12636us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:37.328062 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:37.328906 (+ 844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:37.331200 (+ 2294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:37.331211 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:37.332197 (+ 986us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:37.333041 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:37.333064 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:37.334752 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:37.334757 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:37.336240 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:37.336259 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:37.337165 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:37.337171 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:37.337488 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:37.338293 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:37.338310 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:37.340833 (+ 2523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:37.344029 (+ 3196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:37.344038 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:37.344053 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:37.344631 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:37.345063 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:37.345068 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:37.345072 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:37.345264 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:37.346067 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:37.346075 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:37.346381 (+ 306us) client_negotiation.cc:769] Sending connection context
0808 03:54:37.346621 (+ 240us) client_negotiation.cc:240] Negotiation successful
0808 03:54:37.347163 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12469,"mutex_wait_us":114,"spinlock_wait_cycles":27264,"thread_start_us":12700,"threads_started":1}
I20260808 03:54:37.347786 17467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:37.315388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:34861 (local address 127.26.146.129:54719)
0808 03:54:37.328319 (+ 12931us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:37.328338 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:37.328741 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:37.331201 (+ 2460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:37.331211 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:37.332198 (+ 987us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:37.333054 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:37.333070 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:37.334742 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:37.334746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:37.336239 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:37.336258 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:37.337299 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:37.337307 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:37.337509 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:37.338241 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:37.338277 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:37.341086 (+ 2809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:37.344781 (+ 3695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:37.344786 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:37.344790 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:37.345137 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:37.345562 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:37.345567 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:37.345570 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:37.345719 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:37.346429 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:37.346434 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:37.346604 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 03:54:37.346729 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 03:54:37.347330 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12813,"mutex_wait_us":398,"spinlock_wait_cycles":45568,"thread_start_us":12243,"threads_started":1}
I20260808 03:54:37.348116 17471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:37.315676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:54719 (local address 127.26.146.131:34861)
0808 03:54:37.328629 (+ 12953us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:37.328645 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:37.328697 (+ 52us) server_negotiation.cc:407] Connection header received
0808 03:54:37.330355 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:37.330365 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:37.330742 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:37.331030 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:37.333237 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:37.334555 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:37.336378 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:37.337005 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:37.337274 (+ 269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:37.341343 (+ 4069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:37.341351 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:37.341363 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:37.341398 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:37.344335 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:37.345271 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:37.345277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:37.345282 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:37.345445 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:37.345832 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:37.345836 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:37.345838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:37.346307 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:37.346475 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:37.347025 (+ 550us) server_negotiation.cc:299] Negotiation successful
0808 03:54:37.347369 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12732,"thread_start_us":12631,"threads_started":1}
W20260808 03:54:37.351526 16836 leader_election.cc:341] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 303151904f2a433da55e682e01bf35a8 (127.26.146.130:40337): Not found: Tablet not found: 426fd9bfe482454b8b15d857d8b68597
W20260808 03:54:37.351912 16832 leader_election.cc:341] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 65b52d7e7d524c3983380747aaefc15c (127.26.146.131:34861): Not found: Tablet not found: 426fd9bfe482454b8b15d857d8b68597
I20260808 03:54:37.352030 16832 leader_election.cc:302] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c90351ca171c4990bbd8672aa972a90a; no voters: 303151904f2a433da55e682e01bf35a8, 65b52d7e7d524c3983380747aaefc15c
I20260808 03:54:37.352468 17465 raft_consensus.cc:2726] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 03:54:37.594826 17473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:37.593989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.131:39251)
0808 03:54:37.594435 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:37.594585 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":203,"threads_started":1}
I20260808 03:54:37.700847 17467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:37.700444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.129:59041)
0808 03:54:37.700602 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:37.700690 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 03:54:38.139345 17474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:38.138643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.130:44195)
0808 03:54:38.139110 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:38.139194 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":253,"threads_started":1}
I20260808 03:54:38.596304 17476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:38.595619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.131:59113)
0808 03:54:38.596072 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:38.596150 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":261,"threads_started":1}
I20260808 03:54:38.702468 17477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:38.701691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.129:41259)
0808 03:54:38.702190 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:38.702286 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":208,"threads_started":1}
I20260808 03:54:39.140829 17479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:39.140103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.130:43083)
0808 03:54:39.140571 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:39.140657 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":244,"threads_started":1}
I20260808 03:54:39.481333 17482 raft_consensus.cc:482] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:54:39.481535 17482 raft_consensus.cc:504] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } }
I20260808 03:54:39.482188 17482 leader_election.cc:288] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 303151904f2a433da55e682e01bf35a8 (127.26.146.130:40337), 65b52d7e7d524c3983380747aaefc15c (127.26.146.131:34861)
W20260808 03:54:39.483139 16836 leader_election.cc:341] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 303151904f2a433da55e682e01bf35a8 (127.26.146.130:40337): Not found: Tablet not found: 426fd9bfe482454b8b15d857d8b68597
W20260808 03:54:39.483333 16832 leader_election.cc:341] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 65b52d7e7d524c3983380747aaefc15c (127.26.146.131:34861): Not found: Tablet not found: 426fd9bfe482454b8b15d857d8b68597
I20260808 03:54:39.483465 16832 leader_election.cc:302] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c90351ca171c4990bbd8672aa972a90a; no voters: 303151904f2a433da55e682e01bf35a8, 65b52d7e7d524c3983380747aaefc15c
I20260808 03:54:39.483762 17482 raft_consensus.cc:2726] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 03:54:39.597877 17483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:39.597179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.131:42075)
0808 03:54:39.597629 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:39.597713 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":233,"threads_started":1}
I20260808 03:54:39.704310 17484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:39.703312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.129:40855)
0808 03:54:39.703905 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:39.704070 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":265,"threads_started":1}
I20260808 03:54:40.142396 17485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:40.141666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.130:40069)
0808 03:54:40.142129 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:40.142213 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":207,"threads_started":1}
I20260808 03:54:40.599484 17486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:40.598766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.131:34745)
0808 03:54:40.599222 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:40.599303 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":226,"threads_started":1}
I20260808 03:54:40.706101 17487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:40.705407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.129:44101)
0808 03:54:40.705840 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:40.705918 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":242,"threads_started":1}
I20260808 03:54:41.143934 17488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:41.143173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.130:43363)
0808 03:54:41.143642 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:41.143728 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":243,"threads_started":1}
I20260808 03:54:41.497738 17489 raft_consensus.cc:482] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:54:41.497936 17489 raft_consensus.cc:504] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } }
I20260808 03:54:41.498602 17489 leader_election.cc:288] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 303151904f2a433da55e682e01bf35a8 (127.26.146.130:40337), 65b52d7e7d524c3983380747aaefc15c (127.26.146.131:34861)
W20260808 03:54:41.499557 16832 leader_election.cc:341] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 65b52d7e7d524c3983380747aaefc15c (127.26.146.131:34861): Not found: Tablet not found: 426fd9bfe482454b8b15d857d8b68597
W20260808 03:54:41.499739 16836 leader_election.cc:341] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 303151904f2a433da55e682e01bf35a8 (127.26.146.130:40337): Not found: Tablet not found: 426fd9bfe482454b8b15d857d8b68597
I20260808 03:54:41.499858 16836 leader_election.cc:302] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c90351ca171c4990bbd8672aa972a90a; no voters: 303151904f2a433da55e682e01bf35a8, 65b52d7e7d524c3983380747aaefc15c
I20260808 03:54:41.500074 17489 raft_consensus.cc:2726] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 03:54:41.601059 17490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:41.600319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.131:41071)
0808 03:54:41.600796 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:41.600891 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":250,"threads_started":1}
I20260808 03:54:41.707746 17491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:41.706948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.129:45063)
0808 03:54:41.707437 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:41.707540 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":240,"threads_started":1}
I20260808 03:54:42.145506 17493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:42.144792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.130:56213)
0808 03:54:42.145248 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:42.145328 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":240,"threads_started":1}
I20260808 03:54:42.602574 17495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:42.601877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.131:43045)
0808 03:54:42.602314 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:42.602395 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":233,"threads_started":1}
I20260808 03:54:42.709451 17496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:42.708666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.129:34577)
0808 03:54:42.709130 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:42.709226 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":226,"threads_started":1}
I20260808 03:54:43.147167 17497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:43.146349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.130:58445)
0808 03:54:43.146881 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:43.146977 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":229,"threads_started":1}
I20260808 03:54:43.604446 17498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:43.603541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.131:51525)
0808 03:54:43.604140 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:43.604239 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":261,"threads_started":1}
I20260808 03:54:43.711357 17500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:43.710504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.129:48115)
0808 03:54:43.711053 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:43.711150 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":277,"threads_started":1}
I20260808 03:54:44.148943 17501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:44.148192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.130:37039)
0808 03:54:44.148644 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:44.148767 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":214,"threads_started":1}
I20260808 03:54:44.606010 17503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:44.605304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.131:42975)
0808 03:54:44.605743 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:44.605821 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":214,"threads_started":1}
I20260808 03:54:44.713169 17504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:44.712431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.129:53059)
0808 03:54:44.712900 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:44.712989 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":241,"threads_started":1}
I20260808 03:54:45.150451 17507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:45.149741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.130:49255)
0808 03:54:45.150206 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:45.150295 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":214,"threads_started":1}
I20260808 03:54:45.607600 17508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:45.606866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.131:33973)
0808 03:54:45.607325 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:45.607406 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":233,"threads_started":1}
I20260808 03:54:45.714948 17509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:45.714093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.129:42997)
0808 03:54:45.714685 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:45.714767 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":299,"threads_started":1}
I20260808 03:54:46.152181 17510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:46.151353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.130:55233)
0808 03:54:46.151872 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:46.151957 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":288,"threads_started":1}
I20260808 03:54:46.389341 17512 raft_consensus.cc:482] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:54:46.389533 17512 raft_consensus.cc:504] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } }
I20260808 03:54:46.390161 17512 leader_election.cc:288] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 303151904f2a433da55e682e01bf35a8 (127.26.146.130:40337), 65b52d7e7d524c3983380747aaefc15c (127.26.146.131:34861)
W20260808 03:54:46.391117 16836 leader_election.cc:341] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 303151904f2a433da55e682e01bf35a8 (127.26.146.130:40337): Not found: Tablet not found: 426fd9bfe482454b8b15d857d8b68597
W20260808 03:54:46.391274 16832 leader_election.cc:341] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 65b52d7e7d524c3983380747aaefc15c (127.26.146.131:34861): Not found: Tablet not found: 426fd9bfe482454b8b15d857d8b68597
I20260808 03:54:46.391413 16832 leader_election.cc:302] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c90351ca171c4990bbd8672aa972a90a; no voters: 303151904f2a433da55e682e01bf35a8, 65b52d7e7d524c3983380747aaefc15c
I20260808 03:54:46.391747 17512 raft_consensus.cc:2726] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 03:54:46.609300 17513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:46.608498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.131:55591)
0808 03:54:46.608961 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:46.609079 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":255,"threads_started":1}
I20260808 03:54:46.716717 17515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:46.715829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.129:34231)
0808 03:54:46.716348 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:46.716472 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":228,"threads_started":1}
I20260808 03:54:47.153786 17517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:47.153033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.130:41191)
0808 03:54:47.153534 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:47.153627 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":267,"threads_started":1}
I20260808 03:54:47.610924 17519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:47.610192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.131:45529)
0808 03:54:47.610668 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:47.610756 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":233,"threads_started":1}
I20260808 03:54:47.718320 17520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:47.717472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.129:36239)
0808 03:54:47.717942 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:47.718029 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":273,"threads_started":1}
I20260808 03:54:48.155462 17524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:48.154672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.130:36137)
0808 03:54:48.155178 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:48.155277 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":240,"threads_started":1}
I20260808 03:54:48.612620 17525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:48.611856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.131:38485)
0808 03:54:48.612344 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:48.612424 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":258,"threads_started":1}
I20260808 03:54:48.719945 17526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:48.719183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.129:57023)
0808 03:54:48.719636 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:48.719718 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":217,"threads_started":1}
I20260808 03:54:49.111280 17530 raft_consensus.cc:482] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:54:49.111558 17530 raft_consensus.cc:504] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "303151904f2a433da55e682e01bf35a8" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 40337 } } peers { permanent_uuid: "c90351ca171c4990bbd8672aa972a90a" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 44607 } } peers { permanent_uuid: "65b52d7e7d524c3983380747aaefc15c" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34861 } }
I20260808 03:54:49.112259 17530 leader_election.cc:288] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 303151904f2a433da55e682e01bf35a8 (127.26.146.130:40337), 65b52d7e7d524c3983380747aaefc15c (127.26.146.131:34861)
W20260808 03:54:49.113180 16836 leader_election.cc:341] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 303151904f2a433da55e682e01bf35a8 (127.26.146.130:40337): Not found: Tablet not found: 426fd9bfe482454b8b15d857d8b68597
W20260808 03:54:49.113336 16832 leader_election.cc:341] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 65b52d7e7d524c3983380747aaefc15c (127.26.146.131:34861): Not found: Tablet not found: 426fd9bfe482454b8b15d857d8b68597
I20260808 03:54:49.113488 16832 leader_election.cc:302] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c90351ca171c4990bbd8672aa972a90a; no voters: 303151904f2a433da55e682e01bf35a8, 65b52d7e7d524c3983380747aaefc15c
I20260808 03:54:49.113775 17530 raft_consensus.cc:2726] T 426fd9bfe482454b8b15d857d8b68597 P c90351ca171c4990bbd8672aa972a90a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 03:54:49.157109 17531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:49.156352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.130:58379)
0808 03:54:49.156844 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:49.156939 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":259,"threads_started":1}
I20260808 03:54:49.614230 17532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:49.613475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.131:56447)
0808 03:54:49.613935 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:49.614053 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":235,"threads_started":1}
I20260808 03:54:49.721659 17533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:49.720863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.129:53961)
0808 03:54:49.721335 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:49.721419 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":242,"threads_started":1}
I20260808 03:54:50.158633 17536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:50.157914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.130:58191)
0808 03:54:50.158361 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:50.158443 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":229,"threads_started":1}
I20260808 03:54:50.615943 17537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:50.615102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.131:50283)
0808 03:54:50.615580 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:50.615689 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":258,"threads_started":1}
I20260808 03:54:50.723662 17538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:50.722899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.129:46363)
0808 03:54:50.723375 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:50.723468 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":241,"threads_started":1}
I20260808 03:54:51.160472 17539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:51.159690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.130:35977)
0808 03:54:51.160173 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:51.160306 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":238,"threads_started":1}
I20260808 03:54:51.601894 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 16753
I20260808 03:54:51.617174 17542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:51.603023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48670 (local address 127.26.146.129:44607)
0808 03:54:51.604002 (+ 979us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:51.604025 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:51.604062 (+ 37us) server_negotiation.cc:407] Connection header received
0808 03:54:51.604158 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:51.604170 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:51.604476 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:51.604637 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:51.606134 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:51.607154 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:51.608417 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:51.609018 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:51.609115 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:51.611634 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:51.611646 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:51.611662 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:51.611693 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:51.614585 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:51.615207 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:51.615215 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:51.615217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:51.615354 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:51.615632 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:51.615635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:51.615637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:51.616056 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:51.616313 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:51.616574 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 03:54:51.616772 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":789,"thread_start_us":613,"threads_started":1}
I20260808 03:54:51.617830 17543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:51.617078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38815 (local address 127.26.146.131:54597)
0808 03:54:51.617562 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:51.617665 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:38815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":241,"threads_started":1}
I20260808 03:54:51.618458 17017 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:51.706116 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 16753
I20260808 03:54:51.732915 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 17034
I20260808 03:54:51.745769 17546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:51.733688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48192 (local address 127.26.146.130:40337)
0808 03:54:51.734114 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:51.734118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:51.734135 (+ 17us) server_negotiation.cc:407] Connection header received
0808 03:54:51.734194 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:51.734198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:51.734267 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:51.734352 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:51.735728 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:51.736514 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:51.737791 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:51.738219 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:51.738306 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:51.741045 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:51.741050 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:51.741053 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:51.741086 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:51.743461 (+ 2375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:51.744133 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:51.744137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:51.744140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:51.744284 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:51.744586 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:51.744590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:51.744592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:51.744850 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:51.745058 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:51.745271 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 03:54:51.745432 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":218,"threads_started":1}
I20260808 03:54:51.746495 17247 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:51.829592 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 17034
I20260808 03:54:51.853199 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 17261
I20260808 03:54:51.866495 17548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:51.853995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35010 (local address 127.26.146.131:34861)
0808 03:54:51.854546 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:51.854550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:51.854566 (+ 16us) server_negotiation.cc:407] Connection header received
0808 03:54:51.854625 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:51.854628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:51.854694 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:51.854800 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:51.856299 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:51.857084 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:51.858274 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:51.858720 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:51.858829 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:51.861381 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:51.861386 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:51.861389 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:51.861421 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:51.863968 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:51.864589 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:51.864595 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:51.864598 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:51.864757 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:51.865097 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:51.865102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:51.865106 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:51.865363 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:51.865635 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:51.865905 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 03:54:51.866114 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":280,"threads_started":1}
I20260808 03:54:51.867126 17451 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:51.950593 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 17261
2026-08-08T03:54:51Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24551 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:54:52 dist-test-slave-jlzn krb5kdc[17555](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:54:52 dist-test-slave-jlzn krb5kdc[17555](info): set up 2 sockets
Aug 08 03:54:52 dist-test-slave-jlzn krb5kdc[17555](info): commencing operation
krb5kdc: starting...
W20260808 03:54:53.153977 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.005s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:54:53 dist-test-slave-jlzn krb5kdc[17555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:54:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:54:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:54:53.419068 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:46823
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:46823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:54:53.752170 17576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:53.752445 17576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:53.752573 17576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:53.759707 17576 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 03:54:53.759796 17576 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 03:54:53.759869 17576 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 03:54:53.759941 17576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:54:53.760016 17576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:53.760108 17576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:53.760175 17576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:54:53.760249 17576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:54:53.769842 17576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:46823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:46823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:53.771087 17576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:53.772461 17576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:53.784075 17585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:53.784075 17591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:53.784108 17588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:53.785049 17576 server_base.cc:1034] running on GCE node
I20260808 03:54:53.785832 17576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:53.787174 17576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:53.788419 17576 hybrid_clock.cc:648] HybridClock initialized: now 1786161293788359 us; error 28 us; skew 500 ppm
I20260808 03:54:53.788709 17576 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:53 dist-test-slave-jlzn krb5kdc[17555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161293, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:53.794759 17576 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:54:53.796376 17576 webserver.cc:460] Webserver started at http://127.26.146.190:44071/ using document root <none> and password file <none>
I20260808 03:54:53.797042 17576 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:53.797139 17576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:54:53.797451 17576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:54:53.800745 17576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "829c60c206424008a85d262920ecda5b"
format_stamp: "Formatted at 2026-08-08 03:54:53 on dist-test-slave-jlzn"
server_key: "5fbc48436547da729c026a2cb65b3a28"
server_key_iv: "80f879098004bc0e4de54cf421c8b1b0"
server_key_version: "encryptionkey@0"
I20260808 03:54:53.801753 17576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "829c60c206424008a85d262920ecda5b"
format_stamp: "Formatted at 2026-08-08 03:54:53 on dist-test-slave-jlzn"
server_key: "5fbc48436547da729c026a2cb65b3a28"
server_key_iv: "80f879098004bc0e4de54cf421c8b1b0"
server_key_version: "encryptionkey@0"
I20260808 03:54:53.806891 17576 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260808 03:54:53.810211 17607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:53.811435 17576 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:54:53.811591 17576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "829c60c206424008a85d262920ecda5b"
format_stamp: "Formatted at 2026-08-08 03:54:53 on dist-test-slave-jlzn"
server_key: "5fbc48436547da729c026a2cb65b3a28"
server_key_iv: "80f879098004bc0e4de54cf421c8b1b0"
server_key_version: "encryptionkey@0"
I20260808 03:54:53.811837 17576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:53.844774 17576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:53.848860 17576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:53.849110 17576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:53.866611 17576 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:46823
I20260808 03:54:53.866643 17677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:46823 every 8 connection(s)
I20260808 03:54:53.867950 17576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:54:53.871732 17678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:54:53.873111 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 17576
I20260808 03:54:53.873308 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:54:53.873978 27210 external_mini_cluster.cc:1428] Setting key 759662694f6df058b62840069c711002
I20260808 03:54:53.885761 17678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b: Bootstrap starting.
Aug 08 03:54:53 dist-test-slave-jlzn krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:54:53.889011 17678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b: Neither blocks nor log segments found. Creating new log.
I20260808 03:54:53.890286 17678 log.cc:824] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b: Log is configured to *not* fsync() on all Append() calls
I20260808 03:54:53.893265 17678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b: No bootstrap required, opened a new log
I20260808 03:54:53.897285 17684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:53.875965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46382 (local address 127.26.146.190:46823)
0808 03:54:53.876813 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:53.876828 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:53.876879 (+ 51us) server_negotiation.cc:407] Connection header received
0808 03:54:53.878635 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:53.878669 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:53.879189 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:53.879602 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:53.881029 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:53.883094 (+ 2065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:53.884489 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:53.885499 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:53.885675 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:53.890097 (+ 4422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:53.890105 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:53.890122 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:53.890158 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:53.893216 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:53.894052 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:53.894059 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:53.894064 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:53.894242 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:53.894671 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:53.894674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:53.894676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:53.895195 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:53.895431 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:53.895927 (+ 496us) server_negotiation.cc:299] Negotiation successful
0808 03:54:53.896291 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":336,"threads_started":1}
I20260808 03:54:53.901777 17678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829c60c206424008a85d262920ecda5b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 46823 } }
I20260808 03:54:53.902154 17678 raft_consensus.cc:374] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:54:53.902266 17678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 829c60c206424008a85d262920ecda5b, State: Initialized, Role: FOLLOWER
I20260808 03:54:53.902755 17678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829c60c206424008a85d262920ecda5b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 46823 } }
I20260808 03:54:53.902967 17678 raft_consensus.cc:388] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:54:53.903067 17678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:54:53.903183 17678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:54:53.905478 17678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829c60c206424008a85d262920ecda5b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 46823 } }
I20260808 03:54:53.905920 17678 leader_election.cc:302] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 829c60c206424008a85d262920ecda5b; no voters:
I20260808 03:54:53.906419 17678 leader_election.cc:288] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:54:53.906527 17686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:54:53.907227 17686 raft_consensus.cc:686] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 1 LEADER]: Becoming Leader. State: Replica: 829c60c206424008a85d262920ecda5b, State: Running, Role: LEADER
I20260808 03:54:53.907737 17686 consensus_queue.cc:237] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829c60c206424008a85d262920ecda5b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 46823 } }
I20260808 03:54:53.908492 17678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:54:53.910192 17689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 829c60c206424008a85d262920ecda5b. Latest consensus state: current_term: 1 leader_uuid: "829c60c206424008a85d262920ecda5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829c60c206424008a85d262920ecda5b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 46823 } } }
I20260808 03:54:53.910199 17688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "829c60c206424008a85d262920ecda5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829c60c206424008a85d262920ecda5b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 46823 } } }
I20260808 03:54:53.910671 17689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [sys.catalog]: This master's current role is: LEADER
I20260808 03:54:53.910708 17688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [sys.catalog]: This master's current role is: LEADER
I20260808 03:54:53.916285 17695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:54:53.921696 17695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:54:53.930133 17695 catalog_manager.cc:1347] Generated new cluster ID: e0cd49dfd918475d9fe89200fff654be
I20260808 03:54:53.930251 17695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:54:53.966966 17695 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:54:53.968387 17695 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:54:53.982275 17695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b: Generated new TSK 0
I20260808 03:54:53.983188 17695 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
I20260808 03:54:54.061595 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:46823
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:54:54.392879 17715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:54.393149 17715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:54.393280 17715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:54.400518 17715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:54.400621 17715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:54.400784 17715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:54:54.410380 17715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:46823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:54.411664 17715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:54.412951 17715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:54.425966 17721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:54.425887 17724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:54.425918 17729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:54.426779 17715 server_base.cc:1034] running on GCE node
I20260808 03:54:54.427353 17715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:54.428324 17715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:54.429520 17715 hybrid_clock.cc:648] HybridClock initialized: now 1786161294429466 us; error 36 us; skew 500 ppm
I20260808 03:54:54.429814 17715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:54 dist-test-slave-jlzn krb5kdc[17555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161294, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:54.432945 17715 init.cc:377] Logged in from keytab as kudu/127.26.146.129@KRBTEST.COM (short username kudu)
I20260808 03:54:54.434309 17715 webserver.cc:460] Webserver started at http://127.26.146.129:43807/ using document root <none> and password file <none>
I20260808 03:54:54.434968 17715 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:54.435091 17715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:54:54.435380 17715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:54:54.438335 17715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "ed9d286f2aae46f58fb3a8143e90c959"
format_stamp: "Formatted at 2026-08-08 03:54:54 on dist-test-slave-jlzn"
server_key: "e602046b5269788d3859e78ba7ef8a66"
server_key_iv: "6522547b41972a325f83b492eeb9159a"
server_key_version: "encryptionkey@0"
I20260808 03:54:54.439065 17715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "ed9d286f2aae46f58fb3a8143e90c959"
format_stamp: "Formatted at 2026-08-08 03:54:54 on dist-test-slave-jlzn"
server_key: "e602046b5269788d3859e78ba7ef8a66"
server_key_iv: "6522547b41972a325f83b492eeb9159a"
server_key_version: "encryptionkey@0"
I20260808 03:54:54.443920 17715 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260808 03:54:54.447233 17739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:54.448501 17715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:54:54.448668 17715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "ed9d286f2aae46f58fb3a8143e90c959"
format_stamp: "Formatted at 2026-08-08 03:54:54 on dist-test-slave-jlzn"
server_key: "e602046b5269788d3859e78ba7ef8a66"
server_key_iv: "6522547b41972a325f83b492eeb9159a"
server_key_version: "encryptionkey@0"
I20260808 03:54:54.448845 17715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:54.470335 17715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:54.474084 17715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:54.474344 17715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:54.475248 17715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:54:54.476563 17715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:54:54.476647 17715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:54.476770 17715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:54:54.476824 17715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:54.506809 17715 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:32855
I20260808 03:54:54.506852 17916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:32855 every 8 connection(s)
I20260808 03:54:54.508136 17715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:54:54.517551 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 17715
I20260808 03:54:54.517761 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:54:54.518646 27210 external_mini_cluster.cc:1428] Setting key cc282e41784352a71273cda18dc5a04c
Aug 08 03:54:54 dist-test-slave-jlzn krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161294, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:54:54.528268 17919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:54.510620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:41359 (local address 127.26.146.190:46823)
0808 03:54:54.511047 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:54.511051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:54.512051 (+ 1000us) server_negotiation.cc:407] Connection header received
0808 03:54:54.513846 (+ 1795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:54.513851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:54.513922 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:54.514021 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:54.515502 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:54.516350 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:54.517950 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:54.518627 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:54.518777 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:54.522372 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:54.522378 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:54.522384 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:54.522424 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:54.525370 (+ 2946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:54.526048 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:54.526054 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:54.526057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:54.526206 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:54.526582 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:54.526587 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:54.526590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:54.526916 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:54.527058 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:54.527685 (+ 627us) server_negotiation.cc:299] Negotiation successful
0808 03:54:54.527931 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":200,"threads_started":1}
I20260808 03:54:54.529095 17923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:54.511165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.129:41359)
0808 03:54:54.511893 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:54.511937 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:54.513573 (+ 1636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:54.514212 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:54.514219 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:54.514714 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:54.515317 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:54.515338 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:54.516479 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:54.516483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:54.517803 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:54.517817 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:54.518852 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:54.518858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:54.519048 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:54.519713 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:54.519744 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:54.522149 (+ 2405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:54.525494 (+ 3345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:54.525501 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:54.525513 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:54.525934 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:54.526299 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:54.526302 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:54.526304 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:54.526478 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:54.527017 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:54.527022 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:54.527329 (+ 307us) client_negotiation.cc:769] Sending connection context
0808 03:54:54.527551 (+ 222us) client_negotiation.cc:240] Negotiation successful
0808 03:54:54.527978 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":277,"threads_started":1}
I20260808 03:54:54.530584 17917 heartbeater.cc:347] Connected to a master server at 127.26.146.190:46823
I20260808 03:54:54.530870 17917 heartbeater.cc:464] Registering TS with master...
I20260808 03:54:54.531579 17917 heartbeater.cc:511] Master 127.26.146.190:46823 requested a full tablet report, sending...
I20260808 03:54:54.533587 17628 ts_manager.cc:194] Registered new tserver with Master: ed9d286f2aae46f58fb3a8143e90c959 (127.26.146.129:32855)
I20260808 03:54:54.535245 17628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.129@KRBTEST.COM'} at 127.26.146.129:41359
WARNING: no policy specified for kudu/127.26.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
I20260808 03:54:54.579282 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:46823
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:54:54.915179 17931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:54.915462 17931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:54.915583 17931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:54.922819 17931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:54.922922 17931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:54.923033 17931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:54:54.932552 17931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:46823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:54.933794 17931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:54.935043 17931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:54.947707 17942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:54.947790 17947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:54.947754 17944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:54.948676 17931 server_base.cc:1034] running on GCE node
I20260808 03:54:54.949245 17931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:54.950110 17931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:54.951321 17931 hybrid_clock.cc:648] HybridClock initialized: now 1786161294951269 us; error 47 us; skew 500 ppm
I20260808 03:54:54.951629 17931 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:54 dist-test-slave-jlzn krb5kdc[17555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161294, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:54.954656 17931 init.cc:377] Logged in from keytab as kudu/127.26.146.130@KRBTEST.COM (short username kudu)
I20260808 03:54:54.956043 17931 webserver.cc:460] Webserver started at http://127.26.146.130:41263/ using document root <none> and password file <none>
I20260808 03:54:54.956637 17931 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:54.956748 17931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:54:54.957000 17931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:54:54.959756 17931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "442d2f461aa441ff8d4ce07f5bc7dea7"
format_stamp: "Formatted at 2026-08-08 03:54:54 on dist-test-slave-jlzn"
server_key: "a277e5c3d424f1fc7cdc4131f020f6a5"
server_key_iv: "ada5824d684fa48fb50197a1076d8d11"
server_key_version: "encryptionkey@0"
I20260808 03:54:54.960495 17931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "442d2f461aa441ff8d4ce07f5bc7dea7"
format_stamp: "Formatted at 2026-08-08 03:54:54 on dist-test-slave-jlzn"
server_key: "a277e5c3d424f1fc7cdc4131f020f6a5"
server_key_iv: "ada5824d684fa48fb50197a1076d8d11"
server_key_version: "encryptionkey@0"
I20260808 03:54:54.965296 17931 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 03:54:54.968487 17958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:54.969668 17931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:54:54.969802 17931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "442d2f461aa441ff8d4ce07f5bc7dea7"
format_stamp: "Formatted at 2026-08-08 03:54:54 on dist-test-slave-jlzn"
server_key: "a277e5c3d424f1fc7cdc4131f020f6a5"
server_key_iv: "ada5824d684fa48fb50197a1076d8d11"
server_key_version: "encryptionkey@0"
I20260808 03:54:54.969955 17931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:54.982010 17931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:54.985630 17931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:54.985868 17931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:54.986658 17931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:54:54.987901 17931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:54:54.987975 17931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:54.988058 17931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:54:54.988101 17931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:55.018293 17931 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:35807
I20260808 03:54:55.018332 18121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:35807 every 8 connection(s)
I20260808 03:54:55.019522 17931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:54:55.025352 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 17931
I20260808 03:54:55.025557 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:54:55.026132 27210 external_mini_cluster.cc:1428] Setting key 885dcfe9fe0edbd656f66b1bda0adc8f
Aug 08 03:54:55 dist-test-slave-jlzn krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161294, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:54:55.039907 17919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:55.021931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:35755 (local address 127.26.146.190:46823)
0808 03:54:55.022102 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:55.022107 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:55.023239 (+ 1132us) server_negotiation.cc:407] Connection header received
0808 03:54:55.025017 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:55.025021 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:55.025094 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:55.025189 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:55.027097 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.028088 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.030162 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.030592 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.030684 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:55.034493 (+ 3809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:55.034498 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:55.034501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:55.034534 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:55.037030 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:55.037836 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:55.037840 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:55.037843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:55.037983 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:55.038391 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:55.038394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:55.038396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:55.038634 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:55.038755 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:55.039452 (+ 697us) server_negotiation.cc:299] Negotiation successful
0808 03:54:55.039617 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 03:54:55.040949 18124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:55.022391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.130:35755)
0808 03:54:55.023085 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:55.023122 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:55.024767 (+ 1645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:55.025451 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:55.025462 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:55.026176 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:55.026921 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:55.026942 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.028220 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.028224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:55.030025 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:55.030041 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.030808 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.030815 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:55.031087 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:55.031761 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:55.031786 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:55.034292 (+ 2506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:55.037161 (+ 2869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:55.037169 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:55.037184 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:55.037736 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:55.038079 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:55.038084 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:55.038087 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:55.038308 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:55.038744 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:55.038750 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:55.039125 (+ 375us) client_negotiation.cc:769] Sending connection context
0808 03:54:55.039404 (+ 279us) client_negotiation.cc:240] Negotiation successful
0808 03:54:55.039899 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":276,"threads_started":1}
I20260808 03:54:55.042312 18122 heartbeater.cc:347] Connected to a master server at 127.26.146.190:46823
I20260808 03:54:55.042583 18122 heartbeater.cc:464] Registering TS with master...
I20260808 03:54:55.043288 18122 heartbeater.cc:511] Master 127.26.146.190:46823 requested a full tablet report, sending...
I20260808 03:54:55.044699 17628 ts_manager.cc:194] Registered new tserver with Master: 442d2f461aa441ff8d4ce07f5bc7dea7 (127.26.146.130:35807)
I20260808 03:54:55.045493 17628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.130@KRBTEST.COM'} at 127.26.146.130:35755
WARNING: no policy specified for kudu/127.26.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
I20260808 03:54:55.086108 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:46823
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:54:55.410990 18129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:54:55.411269 18129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:54:55.411414 18129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:54:55.418497 18129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:54:55.418614 18129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:54:55.418772 18129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:54:55.428131 18129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:46823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:54:55.429359 18129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:54:55.430673 18129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:54:55.443323 18139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:55.443405 18136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:54:55.443351 18135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:54:55.444600 18129 server_base.cc:1034] running on GCE node
I20260808 03:54:55.445165 18129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:54:55.446017 18129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:54:55.447206 18129 hybrid_clock.cc:648] HybridClock initialized: now 1786161295447160 us; error 33 us; skew 500 ppm
I20260808 03:54:55.447484 18129 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:54:55 dist-test-slave-jlzn krb5kdc[17555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161295, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:54:55.450572 18129 init.cc:377] Logged in from keytab as kudu/127.26.146.131@KRBTEST.COM (short username kudu)
I20260808 03:54:55.452078 18129 webserver.cc:460] Webserver started at http://127.26.146.131:33611/ using document root <none> and password file <none>
I20260808 03:54:55.452766 18129 fs_manager.cc:359] Metadata directory not provided
I20260808 03:54:55.452899 18129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:54:55.453204 18129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:54:55.456141 18129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "f9ca6c74940547508a2f99ef79259478"
format_stamp: "Formatted at 2026-08-08 03:54:55 on dist-test-slave-jlzn"
server_key: "5f0443d6454e8ca1e5a13f9330ff49b1"
server_key_iv: "aab9fcd3b548cc654fb603d5ec4686e4"
server_key_version: "encryptionkey@0"
I20260808 03:54:55.456875 18129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "f9ca6c74940547508a2f99ef79259478"
format_stamp: "Formatted at 2026-08-08 03:54:55 on dist-test-slave-jlzn"
server_key: "5f0443d6454e8ca1e5a13f9330ff49b1"
server_key_iv: "aab9fcd3b548cc654fb603d5ec4686e4"
server_key_version: "encryptionkey@0"
I20260808 03:54:55.461817 18129 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:54:55.465193 18154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:55.466387 18129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:54:55.466531 18129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "f9ca6c74940547508a2f99ef79259478"
format_stamp: "Formatted at 2026-08-08 03:54:55 on dist-test-slave-jlzn"
server_key: "5f0443d6454e8ca1e5a13f9330ff49b1"
server_key_iv: "aab9fcd3b548cc654fb603d5ec4686e4"
server_key_version: "encryptionkey@0"
I20260808 03:54:55.466723 18129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:54:55.479915 18129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:54:55.483791 18129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:54:55.484086 18129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:54:55.485005 18129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:54:55.486272 18129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:54:55.486349 18129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:55.486469 18129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:54:55.486526 18129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:54:55.517408 18129 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:44625
I20260808 03:54:55.517452 18317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:44625 every 8 connection(s)
I20260808 03:54:55.518631 18129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:54:55.520496 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 18129
I20260808 03:54:55.520668 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:54:55.521157 27210 external_mini_cluster.cc:1428] Setting key 752e69fc6f64a68bcf8b15b91ad5639b
Aug 08 03:54:55 dist-test-slave-jlzn krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161295, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:54:55.537890 17919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:55.521252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:53529 (local address 127.26.146.190:46823)
0808 03:54:55.521461 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:55.521468 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:55.522634 (+ 1166us) server_negotiation.cc:407] Connection header received
0808 03:54:55.524878 (+ 2244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:55.524884 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:55.524978 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:55.525061 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:55.526496 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.527250 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.528801 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.529244 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.529354 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:55.532565 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:55.532570 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:55.532573 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:55.532605 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:55.534911 (+ 2306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:55.535771 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:55.535776 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:55.535779 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:55.536017 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:55.536461 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:55.536466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:55.536470 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:55.536773 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:55.536886 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:55.537480 (+ 594us) server_negotiation.cc:299] Negotiation successful
0808 03:54:55.537638 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 03:54:55.538271 17917 heartbeater.cc:503] Master 127.26.146.190:46823 was elected leader, sending a full tablet report...
I20260808 03:54:55.539340 18323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:55.521705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.131:53529)
0808 03:54:55.522483 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:55.522539 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:55.524625 (+ 2086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:55.525298 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:55.525305 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:55.525804 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:54:55.526339 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:55.526356 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.527395 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.527398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:55.528658 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:55.528671 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.529458 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.529465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:55.529678 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:55.530289 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:54:55.530310 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:54:55.532372 (+ 2062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:54:55.535062 (+ 2690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:55.535071 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:55.535086 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:55.535648 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:55.536125 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:54:55.536130 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:54:55.536134 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:54:55.536330 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:54:55.536891 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:54:55.536897 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:54:55.537201 (+ 304us) client_negotiation.cc:769] Sending connection context
0808 03:54:55.537412 (+ 211us) client_negotiation.cc:240] Negotiation successful
0808 03:54:55.537775 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":294,"threads_started":1}
I20260808 03:54:55.540616 18318 heartbeater.cc:347] Connected to a master server at 127.26.146.190:46823
I20260808 03:54:55.540921 18318 heartbeater.cc:464] Registering TS with master...
I20260808 03:54:55.541742 18318 heartbeater.cc:511] Master 127.26.146.190:46823 requested a full tablet report, sending...
I20260808 03:54:55.543085 17628 ts_manager.cc:194] Registered new tserver with Master: f9ca6c74940547508a2f99ef79259478 (127.26.146.131:44625)
I20260808 03:54:55.543900 17628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.131@KRBTEST.COM'} at 127.26.146.131:53529
I20260808 03:54:55.548552 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:54:55.565156 17919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:55.553044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46398 (local address 127.26.146.190:46823)
0808 03:54:55.553223 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:55.553230 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:55.553646 (+ 416us) server_negotiation.cc:407] Connection header received
0808 03:54:55.553800 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:55.553804 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:55.553865 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:55.553937 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:55.555328 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.556131 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.557544 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.557963 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.558087 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:55.560451 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:55.560455 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:55.560459 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:55.560492 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:55.562916 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:55.563483 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:55.563488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:55.563491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:55.563629 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:55.563959 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:55.563962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:55.563964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:55.564291 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:55.564470 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:55.564693 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 03:54:55.564846 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 03:54:55.568920 17628 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46398:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:54:55.575826 17628 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:54:55.608839 18338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:55.587035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:32855 (local address 127.0.0.1:55434)
0808 03:54:55.599445 (+ 12410us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:55.599502 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:55.599705 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:55.600689 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:55.600697 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:55.600726 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:54:55.601130 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:55.601146 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.603454 (+ 2308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.603460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:55.605916 (+ 2456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:55.605931 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.607837 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.607844 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:55.608048 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:55.608111 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 03:54:55.608235 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 03:54:55.608426 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12120,"thread_start_us":12839,"threads_started":1}
I20260808 03:54:55.609557 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:55.587347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55434 (local address 127.26.146.129:32855)
0808 03:54:55.599991 (+ 12644us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:55.600001 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:55.600044 (+ 43us) server_negotiation.cc:407] Connection header received
0808 03:54:55.600154 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:55.600162 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:55.600425 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:55.601343 (+ 918us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:54:55.601613 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.603287 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.606139 (+ 2852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.607604 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.608761 (+ 1157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:55.608947 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:55.609075 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 03:54:55.609239 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12438,"thread_start_us":12267,"threads_started":1}
I20260808 03:54:55.609995 18339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:55.587216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:44625 (local address 127.0.0.1:40226)
0808 03:54:55.600318 (+ 13102us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:55.600338 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:55.600448 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:55.601096 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:55.601102 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:55.601130 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:54:55.601749 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:55.601762 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.605338 (+ 3576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.605344 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:55.607414 (+ 2070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:55.607431 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.609238 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.609245 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:55.609420 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:55.609451 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 03:54:55.609516 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 03:54:55.609637 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12988,"mutex_wait_us":2,"thread_start_us":12838,"threads_started":1}
I20260808 03:54:55.610668 18342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:55.587821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40226 (local address 127.26.146.131:44625)
0808 03:54:55.599520 (+ 11699us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:55.599529 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:55.600559 (+ 1030us) server_negotiation.cc:407] Connection header received
0808 03:54:55.600667 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:55.600675 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:55.600885 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:55.601956 (+ 1071us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:54:55.602239 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.605140 (+ 2901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.607605 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.608996 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.609966 (+ 970us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:55.610122 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:55.610194 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 03:54:55.610346 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11533,"thread_start_us":11388,"threads_started":1}
I20260808 03:54:55.613224 18343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:55.600938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38152 (local address 127.26.146.130:35807)
0808 03:54:55.604962 (+ 4024us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:55.604971 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:55.605489 (+ 518us) server_negotiation.cc:407] Connection header received
0808 03:54:55.605568 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:55.605573 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:55.605735 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:55.605885 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:54:55.606535 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.607727 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.610416 (+ 2689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.612280 (+ 1864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.612434 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:55.612590 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:55.612795 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 03:54:55.612959 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3864,"thread_start_us":3759,"threads_started":1}
I20260808 03:54:55.613235 18345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:55.601443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:35807 (local address 127.0.0.1:38152)
0808 03:54:55.605271 (+ 3828us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:55.605287 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:55.605389 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:55.605908 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:55.605913 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:55.605935 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:54:55.606367 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:55.606377 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.607881 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.607886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:55.610278 (+ 2392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:55.610290 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.612469 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.612475 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:55.612635 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:55.612663 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 03:54:55.612744 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 03:54:55.612869 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3696,"mutex_wait_us":34,"thread_start_us":3557,"threads_started":1}
I20260808 03:54:55.613940 17815 tablet_service.cc:1467] Processing CreateTablet for tablet 3b0bdf8133e749f9a4c8cdbe5efd448a (DEFAULT_TABLE table=test-table [id=3e31abc3f1424247a99be0eae31cf07e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:54:55.614214 18218 tablet_service.cc:1467] Processing CreateTablet for tablet 3b0bdf8133e749f9a4c8cdbe5efd448a (DEFAULT_TABLE table=test-table [id=3e31abc3f1424247a99be0eae31cf07e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:54:55.615593 17815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b0bdf8133e749f9a4c8cdbe5efd448a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:54:55.615718 18218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b0bdf8133e749f9a4c8cdbe5efd448a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:54:55.616230 18023 tablet_service.cc:1467] Processing CreateTablet for tablet 3b0bdf8133e749f9a4c8cdbe5efd448a (DEFAULT_TABLE table=test-table [id=3e31abc3f1424247a99be0eae31cf07e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:54:55.617324 18023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b0bdf8133e749f9a4c8cdbe5efd448a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:54:55.631193 18349 tablet_bootstrap.cc:492] T 3b0bdf8133e749f9a4c8cdbe5efd448a P ed9d286f2aae46f58fb3a8143e90c959: Bootstrap starting.
I20260808 03:54:55.632850 18348 tablet_bootstrap.cc:492] T 3b0bdf8133e749f9a4c8cdbe5efd448a P f9ca6c74940547508a2f99ef79259478: Bootstrap starting.
I20260808 03:54:55.633360 18346 tablet_bootstrap.cc:492] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7: Bootstrap starting.
I20260808 03:54:55.636339 18349 tablet_bootstrap.cc:654] T 3b0bdf8133e749f9a4c8cdbe5efd448a P ed9d286f2aae46f58fb3a8143e90c959: Neither blocks nor log segments found. Creating new log.
I20260808 03:54:55.636729 18348 tablet_bootstrap.cc:654] T 3b0bdf8133e749f9a4c8cdbe5efd448a P f9ca6c74940547508a2f99ef79259478: Neither blocks nor log segments found. Creating new log.
I20260808 03:54:55.637116 18346 tablet_bootstrap.cc:654] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7: Neither blocks nor log segments found. Creating new log.
I20260808 03:54:55.637437 18349 log.cc:824] T 3b0bdf8133e749f9a4c8cdbe5efd448a P ed9d286f2aae46f58fb3a8143e90c959: Log is configured to *not* fsync() on all Append() calls
I20260808 03:54:55.638302 18346 log.cc:824] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7: Log is configured to *not* fsync() on all Append() calls
I20260808 03:54:55.638689 18348 log.cc:824] T 3b0bdf8133e749f9a4c8cdbe5efd448a P f9ca6c74940547508a2f99ef79259478: Log is configured to *not* fsync() on all Append() calls
I20260808 03:54:55.640640 18349 tablet_bootstrap.cc:492] T 3b0bdf8133e749f9a4c8cdbe5efd448a P ed9d286f2aae46f58fb3a8143e90c959: No bootstrap required, opened a new log
I20260808 03:54:55.640887 18349 ts_tablet_manager.cc:1397] T 3b0bdf8133e749f9a4c8cdbe5efd448a P ed9d286f2aae46f58fb3a8143e90c959: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260808 03:54:55.641140 18346 tablet_bootstrap.cc:492] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7: No bootstrap required, opened a new log
I20260808 03:54:55.641407 18346 ts_tablet_manager.cc:1397] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 03:54:55.641747 18348 tablet_bootstrap.cc:492] T 3b0bdf8133e749f9a4c8cdbe5efd448a P f9ca6c74940547508a2f99ef79259478: No bootstrap required, opened a new log
I20260808 03:54:55.641985 18348 ts_tablet_manager.cc:1397] T 3b0bdf8133e749f9a4c8cdbe5efd448a P f9ca6c74940547508a2f99ef79259478: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 03:54:55.648065 18349 raft_consensus.cc:348] T 3b0bdf8133e749f9a4c8cdbe5efd448a P ed9d286f2aae46f58fb3a8143e90c959 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } }
I20260808 03:54:55.648391 18349 raft_consensus.cc:374] T 3b0bdf8133e749f9a4c8cdbe5efd448a P ed9d286f2aae46f58fb3a8143e90c959 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:54:55.648481 18349 raft_consensus.cc:729] T 3b0bdf8133e749f9a4c8cdbe5efd448a P ed9d286f2aae46f58fb3a8143e90c959 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed9d286f2aae46f58fb3a8143e90c959, State: Initialized, Role: FOLLOWER
I20260808 03:54:55.649047 18349 consensus_queue.cc:260] T 3b0bdf8133e749f9a4c8cdbe5efd448a P ed9d286f2aae46f58fb3a8143e90c959 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } }
I20260808 03:54:55.650751 18349 ts_tablet_manager.cc:1428] T 3b0bdf8133e749f9a4c8cdbe5efd448a P ed9d286f2aae46f58fb3a8143e90c959: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260808 03:54:55.651669 18346 raft_consensus.cc:348] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } }
I20260808 03:54:55.652145 18346 raft_consensus.cc:374] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:54:55.652231 18346 raft_consensus.cc:729] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442d2f461aa441ff8d4ce07f5bc7dea7, State: Initialized, Role: FOLLOWER
I20260808 03:54:55.652069 18348 raft_consensus.cc:348] T 3b0bdf8133e749f9a4c8cdbe5efd448a P f9ca6c74940547508a2f99ef79259478 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } }
I20260808 03:54:55.652529 18348 raft_consensus.cc:374] T 3b0bdf8133e749f9a4c8cdbe5efd448a P f9ca6c74940547508a2f99ef79259478 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:54:55.652633 18348 raft_consensus.cc:729] T 3b0bdf8133e749f9a4c8cdbe5efd448a P f9ca6c74940547508a2f99ef79259478 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9ca6c74940547508a2f99ef79259478, State: Initialized, Role: FOLLOWER
I20260808 03:54:55.652918 18346 consensus_queue.cc:260] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } }
I20260808 03:54:55.653278 18348 consensus_queue.cc:260] T 3b0bdf8133e749f9a4c8cdbe5efd448a P f9ca6c74940547508a2f99ef79259478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } }
I20260808 03:54:55.655153 18348 ts_tablet_manager.cc:1428] T 3b0bdf8133e749f9a4c8cdbe5efd448a P f9ca6c74940547508a2f99ef79259478: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260808 03:54:55.655880 18318 heartbeater.cc:503] Master 127.26.146.190:46823 was elected leader, sending a full tablet report...
I20260808 03:54:55.656630 18346 ts_tablet_manager.cc:1428] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260808 03:54:55.657092 18122 heartbeater.cc:503] Master 127.26.146.190:46823 was elected leader, sending a full tablet report...
I20260808 03:54:55.699221 18358 raft_consensus.cc:482] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:54:55.699498 18358 raft_consensus.cc:504] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } }
I20260808 03:54:55.701296 18358 leader_election.cc:288] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed9d286f2aae46f58fb3a8143e90c959 (127.26.146.129:32855), f9ca6c74940547508a2f99ef79259478 (127.26.146.131:44625)
I20260808 03:54:55.709532 18342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:55.702037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:43797 (local address 127.26.146.131:44625)
0808 03:54:55.702242 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:55.702249 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:55.702670 (+ 421us) server_negotiation.cc:407] Connection header received
0808 03:54:55.702830 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:55.702834 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:55.702897 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:55.702963 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:54:55.703507 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.704733 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.707241 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.708462 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.708624 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:55.708697 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:55.709146 (+ 449us) server_negotiation.cc:299] Negotiation successful
0808 03:54:55.709274 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 03:54:55.709533 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:55.701676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:43811 (local address 127.26.146.129:32855)
0808 03:54:55.701882 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:55.701889 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:55.702310 (+ 421us) server_negotiation.cc:407] Connection header received
0808 03:54:55.702568 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:55.702572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:55.702634 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:55.702723 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:54:55.703326 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.704531 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.707248 (+ 2717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.708485 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.708624 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:55.708697 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:55.709169 (+ 472us) server_negotiation.cc:299] Negotiation successful
0808 03:54:55.709297 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 03:54:55.709715 18360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:55.701454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:32855 (local address 127.26.146.130:43811)
0808 03:54:55.702202 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:55.702223 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:55.702420 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:55.702760 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:55.702766 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:55.702789 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:54:55.703165 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:55.703183 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.704691 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.704695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:55.707080 (+ 2385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:55.707090 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.708798 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.708806 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:55.709029 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:55.709064 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 03:54:55.709137 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 03:54:55.709254 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"mutex_wait_us":57,"thread_start_us":398,"threads_started":1}
I20260808 03:54:55.709758 18362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:55.701787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:44625 (local address 127.26.146.130:43797)
0808 03:54:55.702574 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0808 03:54:55.702588 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 03:54:55.702700 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:54:55.703006 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:54:55.703010 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:54:55.703032 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:54:55.703383 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:55.703392 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.704893 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.704898 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:55.707047 (+ 2149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:54:55.707063 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.708728 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.708735 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:54:55.708938 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:55.709015 (+ 77us) client_negotiation.cc:769] Sending connection context
0808 03:54:55.709116 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 03:54:55.709297 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":20,"thread_start_us":489,"threads_started":1}
I20260808 03:54:55.710896 18252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b0bdf8133e749f9a4c8cdbe5efd448a" candidate_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9ca6c74940547508a2f99ef79259478" is_pre_election: true
I20260808 03:54:55.710896 17846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b0bdf8133e749f9a4c8cdbe5efd448a" candidate_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed9d286f2aae46f58fb3a8143e90c959" is_pre_election: true
I20260808 03:54:55.711464 17846 raft_consensus.cc:2454] T 3b0bdf8133e749f9a4c8cdbe5efd448a P ed9d286f2aae46f58fb3a8143e90c959 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442d2f461aa441ff8d4ce07f5bc7dea7 in term 0.
I20260808 03:54:55.711464 18252 raft_consensus.cc:2454] T 3b0bdf8133e749f9a4c8cdbe5efd448a P f9ca6c74940547508a2f99ef79259478 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442d2f461aa441ff8d4ce07f5bc7dea7 in term 0.
I20260808 03:54:55.712181 17965 leader_election.cc:302] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442d2f461aa441ff8d4ce07f5bc7dea7, f9ca6c74940547508a2f99ef79259478; no voters:
I20260808 03:54:55.712584 18358 raft_consensus.cc:2781] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:54:55.712728 18358 raft_consensus.cc:482] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:54:55.712828 18358 raft_consensus.cc:3037] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:54:55.715070 18358 raft_consensus.cc:504] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } }
I20260808 03:54:55.715641 18358 leader_election.cc:288] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7 [CANDIDATE]: Term 1 election: Requested vote from peers ed9d286f2aae46f58fb3a8143e90c959 (127.26.146.129:32855), f9ca6c74940547508a2f99ef79259478 (127.26.146.131:44625)
I20260808 03:54:55.716138 18252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b0bdf8133e749f9a4c8cdbe5efd448a" candidate_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9ca6c74940547508a2f99ef79259478"
I20260808 03:54:55.716113 17846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b0bdf8133e749f9a4c8cdbe5efd448a" candidate_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed9d286f2aae46f58fb3a8143e90c959"
I20260808 03:54:55.716336 18252 raft_consensus.cc:3037] T 3b0bdf8133e749f9a4c8cdbe5efd448a P f9ca6c74940547508a2f99ef79259478 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:54:55.716328 17846 raft_consensus.cc:3037] T 3b0bdf8133e749f9a4c8cdbe5efd448a P ed9d286f2aae46f58fb3a8143e90c959 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:54:55.718534 17846 raft_consensus.cc:2454] T 3b0bdf8133e749f9a4c8cdbe5efd448a P ed9d286f2aae46f58fb3a8143e90c959 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442d2f461aa441ff8d4ce07f5bc7dea7 in term 1.
I20260808 03:54:55.718554 18252 raft_consensus.cc:2454] T 3b0bdf8133e749f9a4c8cdbe5efd448a P f9ca6c74940547508a2f99ef79259478 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442d2f461aa441ff8d4ce07f5bc7dea7 in term 1.
I20260808 03:54:55.718899 17965 leader_election.cc:302] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442d2f461aa441ff8d4ce07f5bc7dea7, f9ca6c74940547508a2f99ef79259478; no voters:
I20260808 03:54:55.719152 18358 raft_consensus.cc:2781] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:54:55.719640 18358 raft_consensus.cc:686] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7 [term 1 LEADER]: Becoming Leader. State: Replica: 442d2f461aa441ff8d4ce07f5bc7dea7, State: Running, Role: LEADER
I20260808 03:54:55.720145 18358 consensus_queue.cc:237] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } }
I20260808 03:54:55.724505 17628 catalog_manager.cc:5495] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 442d2f461aa441ff8d4ce07f5bc7dea7 (127.26.146.130). New cstate: current_term: 1 leader_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } health_report { overall_health: HEALTHY } } }
W20260808 03:54:55.760514 17918 tablet.cc:2369] T 3b0bdf8133e749f9a4c8cdbe5efd448a P ed9d286f2aae46f58fb3a8143e90c959: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:54:55.766523 18343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:55.761390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38166 (local address 127.26.146.130:35807)
0808 03:54:55.761562 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:55.761566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:55.761585 (+ 19us) server_negotiation.cc:407] Connection header received
0808 03:54:55.761760 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:55.761765 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:55.761853 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:55.761974 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:54:55.762270 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.763027 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.764541 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.764943 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.765073 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:55.765544 (+ 471us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:54:55.766042 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:54:55.766186 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:55.766224 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 03:54:55.766297 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 03:54:55.770236 18322 tablet.cc:2369] T 3b0bdf8133e749f9a4c8cdbe5efd448a P f9ca6c74940547508a2f99ef79259478: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:54:55.771718 18123 tablet.cc:2369] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:54:55.774747 18252 raft_consensus.cc:1261] T 3b0bdf8133e749f9a4c8cdbe5efd448a P f9ca6c74940547508a2f99ef79259478 [term 1 FOLLOWER]: Refusing update from remote peer 442d2f461aa441ff8d4ce07f5bc7dea7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:54:55.774828 17846 raft_consensus.cc:1261] T 3b0bdf8133e749f9a4c8cdbe5efd448a P ed9d286f2aae46f58fb3a8143e90c959 [term 1 FOLLOWER]: Refusing update from remote peer 442d2f461aa441ff8d4ce07f5bc7dea7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:54:55.775765 18363 consensus_queue.cc:1035] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:54:55.776165 18358 consensus_queue.cc:1035] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:54:55.792456 18368 mvcc.cc:205] Tried to move back new op lower bound from 7316116667483205632 to 7316116667273236480. Current Snapshot: MvccSnapshot[applied={T|T < 7316116667483205632}]
I20260808 03:54:55.801419 17628 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:54:55.801709 17628 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:54:55.805295 17628 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b: Sending DeleteTablet for 3 replicas of tablet 3b0bdf8133e749f9a4c8cdbe5efd448a
I20260808 03:54:55.806818 17815 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b0bdf8133e749f9a4c8cdbe5efd448a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:54:55 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:55434
I20260808 03:54:55.807276 18218 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b0bdf8133e749f9a4c8cdbe5efd448a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:54:55 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:40226
I20260808 03:54:55.807626 18023 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b0bdf8133e749f9a4c8cdbe5efd448a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:54:55 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:38152
I20260808 03:54:55.808554 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 17715
I20260808 03:54:55.817963 18380 tablet_replica.cc:330] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7: stopping tablet replica
I20260808 03:54:55.819124 18380 raft_consensus.cc:2229] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7 [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:54:55.820515 18380 raft_consensus.cc:2258] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:54:55.821542 18378 tablet_replica.cc:330] T 3b0bdf8133e749f9a4c8cdbe5efd448a P ed9d286f2aae46f58fb3a8143e90c959: stopping tablet replica
I20260808 03:54:55.822494 18378 raft_consensus.cc:2229] T 3b0bdf8133e749f9a4c8cdbe5efd448a P ed9d286f2aae46f58fb3a8143e90c959 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:54:55.824258 18378 raft_consensus.cc:2258] T 3b0bdf8133e749f9a4c8cdbe5efd448a P ed9d286f2aae46f58fb3a8143e90c959 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:54:55.825215 18380 ts_tablet_manager.cc:1905] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:54:55.827356 18378 ts_tablet_manager.cc:1905] T 3b0bdf8133e749f9a4c8cdbe5efd448a P ed9d286f2aae46f58fb3a8143e90c959: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:54:55.832782 18380 ts_tablet_manager.cc:1918] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:54:55.832955 18380 log.cc:1196] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/3b0bdf8133e749f9a4c8cdbe5efd448a
I20260808 03:54:55.833498 18380 ts_tablet_manager.cc:1939] T 3b0bdf8133e749f9a4c8cdbe5efd448a P 442d2f461aa441ff8d4ce07f5bc7dea7: Deleting consensus metadata
I20260808 03:54:55.833845 18378 ts_tablet_manager.cc:1918] T 3b0bdf8133e749f9a4c8cdbe5efd448a P ed9d286f2aae46f58fb3a8143e90c959: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:54:55.834002 18378 log.cc:1196] T 3b0bdf8133e749f9a4c8cdbe5efd448a P ed9d286f2aae46f58fb3a8143e90c959: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/wal/wals/3b0bdf8133e749f9a4c8cdbe5efd448a
I20260808 03:54:55.834491 18378 ts_tablet_manager.cc:1939] T 3b0bdf8133e749f9a4c8cdbe5efd448a P ed9d286f2aae46f58fb3a8143e90c959: Deleting consensus metadata
I20260808 03:54:55.835408 17610 catalog_manager.cc:4841] TS 442d2f461aa441ff8d4ce07f5bc7dea7 (127.26.146.130:35807): tablet 3b0bdf8133e749f9a4c8cdbe5efd448a (table test-table [id=3e31abc3f1424247a99be0eae31cf07e]) successfully deleted
I20260808 03:54:55.836570 18379 tablet_replica.cc:330] T 3b0bdf8133e749f9a4c8cdbe5efd448a P f9ca6c74940547508a2f99ef79259478: stopping tablet replica
I20260808 03:54:55.836906 17610 catalog_manager.cc:4841] TS ed9d286f2aae46f58fb3a8143e90c959 (127.26.146.129:32855): tablet 3b0bdf8133e749f9a4c8cdbe5efd448a (table test-table [id=3e31abc3f1424247a99be0eae31cf07e]) successfully deleted
I20260808 03:54:55.836989 18379 raft_consensus.cc:2229] T 3b0bdf8133e749f9a4c8cdbe5efd448a P f9ca6c74940547508a2f99ef79259478 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:54:55.837458 18379 raft_consensus.cc:2258] T 3b0bdf8133e749f9a4c8cdbe5efd448a P f9ca6c74940547508a2f99ef79259478 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:54:55.839486 18379 ts_tablet_manager.cc:1905] T 3b0bdf8133e749f9a4c8cdbe5efd448a P f9ca6c74940547508a2f99ef79259478: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 03:54:55 dist-test-slave-jlzn krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.129@KRBTEST.COM
I20260808 03:54:55.844772 18379 ts_tablet_manager.cc:1918] T 3b0bdf8133e749f9a4c8cdbe5efd448a P f9ca6c74940547508a2f99ef79259478: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:54:55.844937 18379 log.cc:1196] T 3b0bdf8133e749f9a4c8cdbe5efd448a P f9ca6c74940547508a2f99ef79259478: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/3b0bdf8133e749f9a4c8cdbe5efd448a
I20260808 03:54:55.845435 18379 ts_tablet_manager.cc:1939] T 3b0bdf8133e749f9a4c8cdbe5efd448a P f9ca6c74940547508a2f99ef79259478: Deleting consensus metadata
I20260808 03:54:55.846889 17612 catalog_manager.cc:4841] TS f9ca6c74940547508a2f99ef79259478 (127.26.146.131:44625): tablet 3b0bdf8133e749f9a4c8cdbe5efd448a (table test-table [id=3e31abc3f1424247a99be0eae31cf07e]) successfully deleted
I20260808 03:54:55.852483 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:55.809768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55440 (local address 127.26.146.129:32855)
0808 03:54:55.809925 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:55.809931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:55.823031 (+ 13100us) server_negotiation.cc:407] Connection header received
0808 03:54:55.823536 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:55.823542 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:55.823638 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:55.823726 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:55.836126 (+ 12400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.837213 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.839105 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.839695 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.839791 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:55.845510 (+ 5719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:55.845527 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:55.845547 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:55.845592 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:55.849374 (+ 3782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:55.850107 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:55.850116 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:55.850120 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:55.850292 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:55.850649 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:55.850654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:55.850657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:55.851123 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:55.851422 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:55.851713 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 03:54:55.852073 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 03:54:55.853327 17911 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:55.955480 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 17715
I20260808 03:54:55.985008 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 17931
Aug 08 03:54:55 dist-test-slave-jlzn krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.130@KRBTEST.COM
I20260808 03:54:56.002512 18343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:55.985841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38172 (local address 127.26.146.130:35807)
0808 03:54:55.986017 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:55.986021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:55.986096 (+ 75us) server_negotiation.cc:407] Connection header received
0808 03:54:55.986309 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:55.986313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:55.986377 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:55.986460 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:55.988240 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.989022 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.990959 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:55.991399 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:55.991498 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:55.996824 (+ 5326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:55.996836 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:55.996851 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:55.996892 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:55.999935 (+ 3043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:56.000671 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:56.000678 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:56.000680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:56.000817 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:56.001168 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:56.001171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:56.001173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:56.001525 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:56.001775 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:56.002061 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 03:54:56.002228 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 03:54:56.003137 18116 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:56.113407 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 17931
I20260808 03:54:56.133327 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 18129
Aug 08 03:54:56 dist-test-slave-jlzn krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.131@KRBTEST.COM
I20260808 03:54:56.148321 18342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:54:56.134048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40234 (local address 127.26.146.131:44625)
0808 03:54:56.134251 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 03:54:56.134257 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:54:56.134278 (+ 21us) server_negotiation.cc:407] Connection header received
0808 03:54:56.134449 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:54:56.134455 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:54:56.134531 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:54:56.134615 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:54:56.135944 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:56.136680 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:56.137990 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:54:56.138392 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:54:56.138501 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:54:56.142979 (+ 4478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:54:56.142994 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:54:56.143011 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:54:56.143043 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:54:56.145604 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:56.146292 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:56.146299 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:56.146301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:56.146431 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:54:56.146786 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:54:56.146790 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:54:56.146792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:54:56.147134 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:54:56.147424 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0808 03:54:56.147720 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 03:54:56.147985 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 03:54:56.148943 18312 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:56.235100 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 18129
I20260808 03:54:56.254907 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 17576
I20260808 03:54:56.256090 17672 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:54:56.365005 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 17576
I20260808 03:55:16.393721 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:46823
--webserver_interface=127.26.146.190
--webserver_port=44071
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:46823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:55:16.742285 18389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:55:16.742645 18389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:55:16.742792 18389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:55:16.750082 18389 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 03:55:16.750195 18389 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 03:55:16.750258 18389 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 03:55:16.750334 18389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:55:16.750414 18389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:55:16.750515 18389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:55:16.750578 18389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:55:16.750653 18389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:55:16.760763 18389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:46823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:46823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=44071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:55:16.762342 18389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:55:16.763748 18389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:55:16.788031 18410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:55:16.788008 18406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:55:16.788148 18407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:55:16.788303 18389 server_base.cc:1034] running on GCE node
I20260808 03:55:16.789517 18389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:55:16.791119 18389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:55:16.792357 18389 hybrid_clock.cc:648] HybridClock initialized: now 1786161316792281 us; error 46 us; skew 500 ppm
I20260808 03:55:16.792745 18389 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:55:16 dist-test-slave-jlzn krb5kdc[17555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161316, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:55:16.799377 18389 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:55:16.801031 18389 webserver.cc:460] Webserver started at http://127.26.146.190:44071/ using document root <none> and password file <none>
I20260808 03:55:16.801807 18389 fs_manager.cc:359] Metadata directory not provided
I20260808 03:55:16.801934 18389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:55:16.807934 18389 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 03:55:16.810822 18420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:16.812227 18389 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:55:16.812392 18389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "829c60c206424008a85d262920ecda5b"
format_stamp: "Formatted at 2026-08-08 03:54:53 on dist-test-slave-jlzn"
server_key: "5fbc48436547da729c026a2cb65b3a28"
server_key_iv: "80f879098004bc0e4de54cf421c8b1b0"
server_key_version: "encryptionkey@0"
I20260808 03:55:16.813460 18389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:55:16.834426 18389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:55:16.839092 18389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:55:16.839395 18389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:55:16.858359 18389 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:46823
I20260808 03:55:16.858390 18506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:46823 every 8 connection(s)
I20260808 03:55:16.860046 18389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:55:16.865033 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 18389
I20260808 03:55:16.866076 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:32855
--local_ip_for_outbound_sockets=127.26.146.129
--tserver_master_addrs=127.26.146.190:46823
--webserver_port=43807
--webserver_interface=127.26.146.129
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:55:16.868870 18508 sys_catalog.cc:263] Verifying existing consensus state
I20260808 03:55:16.873433 18508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b: Bootstrap starting.
I20260808 03:55:16.895236 18508 log.cc:824] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b: Log is configured to *not* fsync() on all Append() calls
I20260808 03:55:16.908108 18508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 03:55:16.908627 18508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b: Bootstrap complete.
I20260808 03:55:16.917982 18508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829c60c206424008a85d262920ecda5b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 46823 } }
I20260808 03:55:16.918893 18508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 829c60c206424008a85d262920ecda5b, State: Initialized, Role: FOLLOWER
I20260808 03:55:16.919551 18508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829c60c206424008a85d262920ecda5b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 46823 } }
I20260808 03:55:16.919847 18508 raft_consensus.cc:388] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:55:16.919981 18508 raft_consensus.cc:482] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:55:16.920161 18508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 1 FOLLOWER]: Advancing to term 2
I20260808 03:55:16.923982 18508 raft_consensus.cc:504] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829c60c206424008a85d262920ecda5b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 46823 } }
I20260808 03:55:16.924511 18508 leader_election.cc:302] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 829c60c206424008a85d262920ecda5b; no voters:
I20260808 03:55:16.925169 18508 leader_election.cc:288] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 03:55:16.925328 18515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 2 FOLLOWER]: Leader election won for term 2
I20260808 03:55:16.926424 18515 raft_consensus.cc:686] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 2 LEADER]: Becoming Leader. State: Replica: 829c60c206424008a85d262920ecda5b, State: Running, Role: LEADER
I20260808 03:55:16.926975 18515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829c60c206424008a85d262920ecda5b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 46823 } }
I20260808 03:55:16.927516 18508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:55:16.929580 18520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 829c60c206424008a85d262920ecda5b. Latest consensus state: current_term: 2 leader_uuid: "829c60c206424008a85d262920ecda5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829c60c206424008a85d262920ecda5b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 46823 } } }
I20260808 03:55:16.929643 18516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "829c60c206424008a85d262920ecda5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829c60c206424008a85d262920ecda5b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 46823 } } }
I20260808 03:55:16.930087 18520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [sys.catalog]: This master's current role is: LEADER
I20260808 03:55:16.930197 18516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [sys.catalog]: This master's current role is: LEADER
I20260808 03:55:16.944679 18523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:55:16.953089 18523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:55:16.954622 18523 catalog_manager.cc:1259] Loaded cluster ID: e0cd49dfd918475d9fe89200fff654be
I20260808 03:55:16.954736 18523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:55:16.958765 18523 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:55:16.960330 18523 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b: Loaded TSK: 0
I20260808 03:55:16.996879 18523 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b: Generated new TSK 1
I20260808 03:55:16.997336 18523 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 03:55:17.228832 18510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:55:17.229132 18510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:55:17.229285 18510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:55:17.236549 18510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:55:17.236672 18510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:55:17.236819 18510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:55:17.246764 18510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:32855
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=43807
--tserver_master_addrs=127.26.146.190:46823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:55:17.248121 18510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:55:17.249449 18510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:55:17.264257 18558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:55:17.264541 18556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:55:17.264698 18561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:55:17.265164 18510 server_base.cc:1034] running on GCE node
I20260808 03:55:17.265897 18510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:55:17.267098 18510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:55:17.268345 18510 hybrid_clock.cc:648] HybridClock initialized: now 1786161317268290 us; error 38 us; skew 500 ppm
I20260808 03:55:17.268698 18510 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:55:17 dist-test-slave-jlzn krb5kdc[17555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161317, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:55:17.272598 18510 init.cc:377] Logged in from keytab as kudu/127.26.146.129@KRBTEST.COM (short username kudu)
I20260808 03:55:17.274252 18510 webserver.cc:460] Webserver started at http://127.26.146.129:43807/ using document root <none> and password file <none>
I20260808 03:55:17.275100 18510 fs_manager.cc:359] Metadata directory not provided
I20260808 03:55:17.275264 18510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:55:17.282064 18510 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260808 03:55:17.285117 18576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:17.286608 18510 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:55:17.286774 18510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "ed9d286f2aae46f58fb3a8143e90c959"
format_stamp: "Formatted at 2026-08-08 03:54:54 on dist-test-slave-jlzn"
server_key: "e602046b5269788d3859e78ba7ef8a66"
server_key_iv: "6522547b41972a325f83b492eeb9159a"
server_key_version: "encryptionkey@0"
I20260808 03:55:17.288091 18510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:55:17.302349 18510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:55:17.307433 18510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:55:17.307717 18510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:55:17.308868 18510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:55:17.310525 18510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:55:17.310635 18510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:17.310737 18510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:55:17.310818 18510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:17.348388 18510 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:32855
I20260808 03:55:17.348474 18754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:32855 every 8 connection(s)
I20260808 03:55:17.349905 18510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:55:17.357759 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 18510
I20260808 03:55:17.358559 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:35807
--local_ip_for_outbound_sockets=127.26.146.130
--tserver_master_addrs=127.26.146.190:46823
--webserver_port=41263
--webserver_interface=127.26.146.130
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 03:55:17 dist-test-slave-jlzn krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161317, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:55:17.374053 18757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:17.352920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:32937 (local address 127.26.146.190:46823)
0808 03:55:17.353493 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0808 03:55:17.353514 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 03:55:17.353944 (+ 430us) server_negotiation.cc:407] Connection header received
0808 03:55:17.356047 (+ 2103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:55:17.356060 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:55:17.356373 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:55:17.356668 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:55:17.358063 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:17.359923 (+ 1860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:17.361553 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:17.362477 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:17.362655 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:17.366299 (+ 3644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:55:17.366309 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:55:17.366327 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:55:17.366370 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:55:17.369761 (+ 3391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:17.370460 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:17.370469 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:17.370477 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:17.370689 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:17.371059 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:17.371064 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:17.371066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:17.371750 (+ 684us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:55:17.371982 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 03:55:17.372817 (+ 835us) server_negotiation.cc:299] Negotiation successful
0808 03:55:17.373128 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":253,"threads_started":1}
I20260808 03:55:17.374401 18761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:17.352977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.129:32937)
0808 03:55:17.353768 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:17.353814 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 03:55:17.355501 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:55:17.356835 (+ 1334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:55:17.356842 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:55:17.357329 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:55:17.357854 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:55:17.357879 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:55:17.360099 (+ 2220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:55:17.360104 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:55:17.361371 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:55:17.361385 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:55:17.362755 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:55:17.362761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:55:17.362952 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:17.363610 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:55:17.363632 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:55:17.366026 (+ 2394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:55:17.369893 (+ 3867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:55:17.369900 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:55:17.369911 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:55:17.370327 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:55:17.370789 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:55:17.370792 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:55:17.370794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:55:17.370962 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:55:17.371942 (+ 980us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:55:17.371949 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:55:17.372350 (+ 401us) client_negotiation.cc:769] Sending connection context
0808 03:55:17.372647 (+ 297us) client_negotiation.cc:240] Negotiation successful
0808 03:55:17.373176 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":312,"threads_started":1}
I20260808 03:55:17.377135 18755 heartbeater.cc:347] Connected to a master server at 127.26.146.190:46823
I20260808 03:55:17.377421 18755 heartbeater.cc:464] Registering TS with master...
I20260808 03:55:17.378103 18755 heartbeater.cc:511] Master 127.26.146.190:46823 requested a full tablet report, sending...
I20260808 03:55:17.380260 18447 ts_manager.cc:194] Registered new tserver with Master: ed9d286f2aae46f58fb3a8143e90c959 (127.26.146.129:32855)
I20260808 03:55:17.381737 18447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.129@KRBTEST.COM'} at 127.26.146.129:32937
W20260808 03:55:17.699679 18764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:55:17.700004 18764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:55:17.700150 18764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:55:17.707202 18764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:55:17.707324 18764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:55:17.707482 18764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:55:17.717054 18764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:35807
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=41263
--tserver_master_addrs=127.26.146.190:46823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:55:17.718384 18764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:55:17.719743 18764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:55:17.732820 18772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:55:17.732795 18773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:55:17.732861 18776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:55:17.733625 18764 server_base.cc:1034] running on GCE node
I20260808 03:55:17.734212 18764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:55:17.735085 18764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:55:17.736287 18764 hybrid_clock.cc:648] HybridClock initialized: now 1786161317736241 us; error 38 us; skew 500 ppm
I20260808 03:55:17.736573 18764 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:55:17 dist-test-slave-jlzn krb5kdc[17555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161317, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:55:17.739568 18764 init.cc:377] Logged in from keytab as kudu/127.26.146.130@KRBTEST.COM (short username kudu)
I20260808 03:55:17.740967 18764 webserver.cc:460] Webserver started at http://127.26.146.130:41263/ using document root <none> and password file <none>
I20260808 03:55:17.741628 18764 fs_manager.cc:359] Metadata directory not provided
I20260808 03:55:17.741760 18764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:55:17.746891 18764 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260808 03:55:17.749483 18785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:17.750805 18764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260808 03:55:17.750958 18764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "442d2f461aa441ff8d4ce07f5bc7dea7"
format_stamp: "Formatted at 2026-08-08 03:54:54 on dist-test-slave-jlzn"
server_key: "a277e5c3d424f1fc7cdc4131f020f6a5"
server_key_iv: "ada5824d684fa48fb50197a1076d8d11"
server_key_version: "encryptionkey@0"
I20260808 03:55:17.751977 18764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:55:17.769433 18764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:55:17.773072 18764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:55:17.773305 18764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:55:17.774180 18764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:55:17.775421 18764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:55:17.775506 18764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:17.775586 18764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:55:17.775643 18764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:17.806214 18764 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:35807
I20260808 03:55:17.806273 18956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:35807 every 8 connection(s)
I20260808 03:55:17.807632 18764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:55:17.810492 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 18764
I20260808 03:55:17.811228 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:44625
--local_ip_for_outbound_sockets=127.26.146.131
--tserver_master_addrs=127.26.146.190:46823
--webserver_port=33611
--webserver_interface=127.26.146.131
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 03:55:17 dist-test-slave-jlzn krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161317, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:55:17.829291 18757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:17.810136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:40797 (local address 127.26.146.190:46823)
0808 03:55:17.810359 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 03:55:17.810365 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:55:17.811648 (+ 1283us) server_negotiation.cc:407] Connection header received
0808 03:55:17.814068 (+ 2420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:55:17.814072 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:55:17.814144 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:55:17.814232 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:55:17.816147 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:17.816870 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:17.818995 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:17.819451 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:17.819574 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:17.823499 (+ 3925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:55:17.823504 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:55:17.823508 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:55:17.823542 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:55:17.826273 (+ 2731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:17.826994 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:17.827000 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:17.827003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:17.827174 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:17.827569 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:17.827573 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:17.827576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:17.827949 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:55:17.828084 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 03:55:17.828749 (+ 665us) server_negotiation.cc:299] Negotiation successful
0808 03:55:17.828971 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 03:55:17.830188 18962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:17.810608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.130:40797)
0808 03:55:17.811459 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:17.811524 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 03:55:17.813768 (+ 2244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:55:17.814484 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:55:17.814494 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:55:17.815178 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:55:17.815950 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:55:17.815976 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:55:17.817025 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:55:17.817029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:55:17.818840 (+ 1811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:55:17.818855 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:55:17.819669 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:55:17.819676 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:55:17.819997 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:17.820657 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:55:17.820695 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:55:17.823246 (+ 2551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:55:17.826427 (+ 3181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:55:17.826434 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:55:17.826446 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:55:17.826858 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:55:17.827266 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:55:17.827269 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:55:17.827271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:55:17.827445 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:55:17.828061 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:55:17.828067 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:55:17.828387 (+ 320us) client_negotiation.cc:769] Sending connection context
0808 03:55:17.828602 (+ 215us) client_negotiation.cc:240] Negotiation successful
0808 03:55:17.829079 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":297,"threads_started":1}
I20260808 03:55:17.831569 18958 heartbeater.cc:347] Connected to a master server at 127.26.146.190:46823
I20260808 03:55:17.831921 18958 heartbeater.cc:464] Registering TS with master...
I20260808 03:55:17.832795 18958 heartbeater.cc:511] Master 127.26.146.190:46823 requested a full tablet report, sending...
I20260808 03:55:17.834266 18447 ts_manager.cc:194] Registered new tserver with Master: 442d2f461aa441ff8d4ce07f5bc7dea7 (127.26.146.130:35807)
I20260808 03:55:17.835069 18447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.130@KRBTEST.COM'} at 127.26.146.130:40797
W20260808 03:55:18.151252 18963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:55:18.151546 18963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:55:18.151685 18963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:55:18.158851 18963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:55:18.158960 18963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:55:18.159111 18963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:55:18.168603 18963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:44625
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=33611
--tserver_master_addrs=127.26.146.190:46823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:55:18.169840 18963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:55:18.171121 18963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:55:18.183703 18969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:55:18.183729 18970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:55:18.183846 18977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:55:18.184590 18963 server_base.cc:1034] running on GCE node
I20260808 03:55:18.185158 18963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:55:18.186044 18963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:55:18.187250 18963 hybrid_clock.cc:648] HybridClock initialized: now 1786161318187204 us; error 30 us; skew 500 ppm
I20260808 03:55:18.187546 18963 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:55:18 dist-test-slave-jlzn krb5kdc[17555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161318, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:55:18.190507 18963 init.cc:377] Logged in from keytab as kudu/127.26.146.131@KRBTEST.COM (short username kudu)
I20260808 03:55:18.191949 18963 webserver.cc:460] Webserver started at http://127.26.146.131:33611/ using document root <none> and password file <none>
I20260808 03:55:18.192601 18963 fs_manager.cc:359] Metadata directory not provided
I20260808 03:55:18.192737 18963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:55:18.197854 18963 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 03:55:18.200443 18987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:18.201671 18963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:55:18.201814 18963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "f9ca6c74940547508a2f99ef79259478"
format_stamp: "Formatted at 2026-08-08 03:54:55 on dist-test-slave-jlzn"
server_key: "5f0443d6454e8ca1e5a13f9330ff49b1"
server_key_iv: "aab9fcd3b548cc654fb603d5ec4686e4"
server_key_version: "encryptionkey@0"
I20260808 03:55:18.202732 18963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:55:18.237489 18963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:55:18.241438 18963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:55:18.241709 18963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:55:18.242690 18963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:55:18.244026 18963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:55:18.244133 18963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:18.244257 18963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:55:18.244334 18963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:18.275477 18963 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:44625
I20260808 03:55:18.275521 19169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:44625 every 8 connection(s)
I20260808 03:55:18.276945 18963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:55:18.284327 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 18963
Aug 08 03:55:18 dist-test-slave-jlzn krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161318, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:55:18.297549 18757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.279345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:56101 (local address 127.26.146.190:46823)
0808 03:55:18.279519 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 03:55:18.279524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:55:18.280699 (+ 1175us) server_negotiation.cc:407] Connection header received
0808 03:55:18.282438 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:55:18.282443 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:55:18.282513 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:55:18.282601 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:55:18.284074 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.284938 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.287130 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.287540 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.287629 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:18.291507 (+ 3878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:55:18.291512 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:55:18.291515 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:55:18.291559 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:55:18.294247 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:18.294964 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:18.294970 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:18.294972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:18.295094 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:18.295507 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:18.295511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:18.295513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:18.295977 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:55:18.296129 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 03:55:18.297049 (+ 920us) server_negotiation.cc:299] Negotiation successful
0808 03:55:18.297279 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55,"spinlock_wait_cycles":9856}
I20260808 03:55:18.298730 19174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.279820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.131:56101)
0808 03:55:18.280543 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:18.280579 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 03:55:18.282188 (+ 1609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:55:18.282785 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:55:18.282793 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:55:18.283309 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:55:18.283916 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:55:18.283943 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.285136 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.285151 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:55:18.286996 (+ 1845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:55:18.287012 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.287766 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.287774 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:55:18.288093 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:18.288821 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:55:18.288843 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:55:18.291297 (+ 2454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:55:18.294392 (+ 3095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:55:18.294401 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:55:18.294418 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:55:18.294848 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:55:18.295202 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:55:18.295207 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:55:18.295211 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:55:18.295390 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:55:18.296206 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:55:18.296215 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:55:18.296635 (+ 420us) client_negotiation.cc:769] Sending connection context
0808 03:55:18.296958 (+ 323us) client_negotiation.cc:240] Negotiation successful
0808 03:55:18.297468 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":268,"threads_started":1}
I20260808 03:55:18.300205 19176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.285244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57352 (local address 127.26.146.190:46823)
0808 03:55:18.285723 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0808 03:55:18.285728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:55:18.286079 (+ 351us) server_negotiation.cc:407] Connection header received
0808 03:55:18.286273 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:55:18.286277 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:55:18.286342 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:55:18.286422 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:55:18.288451 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.289229 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.290966 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.291378 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.291500 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:18.294316 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:55:18.294321 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:55:18.294324 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:55:18.294355 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:55:18.297468 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:18.298212 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:18.298217 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:18.298220 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:18.298356 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:18.298690 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:18.298695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:18.298698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:18.299018 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:55:18.299275 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0808 03:55:18.299607 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 03:55:18.299844 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"spinlock_wait_cycles":1408,"thread_start_us":223,"threads_started":1}
I20260808 03:55:18.300308 19170 heartbeater.cc:347] Connected to a master server at 127.26.146.190:46823
I20260808 03:55:18.300601 19170 heartbeater.cc:464] Registering TS with master...
I20260808 03:55:18.301308 19170 heartbeater.cc:511] Master 127.26.146.190:46823 requested a full tablet report, sending...
I20260808 03:55:18.302631 18447 ts_manager.cc:194] Registered new tserver with Master: f9ca6c74940547508a2f99ef79259478 (127.26.146.131:44625)
I20260808 03:55:18.303546 18447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.131@KRBTEST.COM'} at 127.26.146.131:56101
I20260808 03:55:18.304383 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:55:18.314661 19176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.305369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57356 (local address 127.26.146.190:46823)
0808 03:55:18.305556 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 03:55:18.305561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:55:18.309397 (+ 3836us) server_negotiation.cc:407] Connection header received
0808 03:55:18.309602 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:55:18.309606 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:55:18.309765 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:55:18.309892 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:55:18.310329 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.310996 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.312542 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.313156 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.313278 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:18.313774 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:55:18.314029 (+ 255us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 03:55:18.314382 (+ 353us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57356: token expired
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 03:55:18.314771 19176 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57356: token expired
W20260808 03:55:18.315335 18327 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.26.146.190:46823)
I20260808 03:55:18.327646 19176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.315876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57366 (local address 127.26.146.190:46823)
0808 03:55:18.316025 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 03:55:18.316029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:55:18.316044 (+ 15us) server_negotiation.cc:407] Connection header received
0808 03:55:18.316133 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:55:18.316143 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:55:18.316196 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:55:18.316271 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:55:18.317627 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.318388 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.319657 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.320080 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.320170 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:18.322599 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:55:18.322603 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:55:18.322606 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:55:18.322629 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:55:18.325245 (+ 2616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:18.325895 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:18.325900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:18.325903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:18.326060 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:18.326478 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:18.326481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:18.326483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:18.326794 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:55:18.326971 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 03:55:18.327212 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 03:55:18.327373 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 03:55:18.331173 18446 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57366:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:55:18.338542 18446 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:55:18.365582 19185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.350075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:32855 (local address 127.0.0.1:41720)
0808 03:55:18.355358 (+ 5283us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:18.355450 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0808 03:55:18.355606 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:55:18.357488 (+ 1882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:55:18.357497 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:55:18.357531 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:55:18.358009 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:55:18.358022 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.360286 (+ 2264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.360293 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:55:18.362966 (+ 2673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:55:18.362982 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.364647 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.364654 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:55:18.364833 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:18.364885 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 03:55:18.365026 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 03:55:18.365220 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4986,"thread_start_us":5363,"threads_started":1}
I20260808 03:55:18.365679 19186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.350565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:44625 (local address 127.0.0.1:32830)
0808 03:55:18.357095 (+ 6530us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:18.357109 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 03:55:18.357234 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:55:18.358120 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:55:18.358125 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:55:18.358151 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:55:18.358536 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:55:18.358552 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.360702 (+ 2150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.360709 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:55:18.363004 (+ 2295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:55:18.363020 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.364975 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.364981 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:55:18.365166 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:18.365197 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 03:55:18.365282 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 03:55:18.365401 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6427,"thread_start_us":6280,"threads_started":1}
I20260808 03:55:18.366516 19187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.350779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41720 (local address 127.26.146.129:32855)
0808 03:55:18.356848 (+ 6069us) server_negotiation.cc:206] Beginning negotiation
0808 03:55:18.356857 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:55:18.356887 (+ 30us) server_negotiation.cc:407] Connection header received
0808 03:55:18.356985 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:55:18.356994 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:55:18.357240 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:55:18.358221 (+ 981us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:55:18.358436 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.360115 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.363174 (+ 3059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.364418 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.364603 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:18.364788 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 03:55:18.366153 (+ 1365us) server_negotiation.cc:299] Negotiation successful
0808 03:55:18.366286 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5875,"thread_start_us":5717,"threads_started":1}
I20260808 03:55:18.366529 19188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.351446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32830 (local address 127.26.146.131:44625)
0808 03:55:18.356347 (+ 4901us) server_negotiation.cc:206] Beginning negotiation
0808 03:55:18.356357 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:55:18.357211 (+ 854us) server_negotiation.cc:407] Connection header received
0808 03:55:18.357391 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:55:18.357399 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:55:18.357624 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:55:18.357833 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:55:18.358760 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.360513 (+ 1753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.363189 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.364686 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.364879 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:18.365055 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 03:55:18.365337 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 03:55:18.365519 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4660,"thread_start_us":4503,"threads_started":1}
I20260808 03:55:18.370453 19067 tablet_service.cc:1467] Processing CreateTablet for tablet d74b5f113f374568ac6565a172189b36 (DEFAULT_TABLE table=test-table [id=458992145b6745a695bf663bc3eb8257]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:55:18.370452 18655 tablet_service.cc:1467] Processing CreateTablet for tablet d74b5f113f374568ac6565a172189b36 (DEFAULT_TABLE table=test-table [id=458992145b6745a695bf663bc3eb8257]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:55:18.372010 19067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d74b5f113f374568ac6565a172189b36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:55:18.372028 18655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d74b5f113f374568ac6565a172189b36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:55:18.372493 19185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.357496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:35807 (local address 127.0.0.1:48470)
0808 03:55:18.365848 (+ 8352us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:18.365867 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 03:55:18.365976 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:55:18.367428 (+ 1452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:55:18.367432 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:55:18.367456 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:55:18.367625 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:55:18.367636 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.369202 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.369207 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:55:18.370359 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:55:18.370371 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.371665 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.371671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:55:18.371873 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:18.371905 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 03:55:18.371984 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 03:55:18.372234 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8246,"thread_start_us":8465,"threads_started":1}
I20260808 03:55:18.372514 19189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.356489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48470 (local address 127.26.146.130:35807)
0808 03:55:18.363920 (+ 7431us) server_negotiation.cc:206] Beginning negotiation
0808 03:55:18.363929 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:55:18.365946 (+ 2017us) server_negotiation.cc:407] Connection header received
0808 03:55:18.366368 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:55:18.366376 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:55:18.366613 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:55:18.367968 (+ 1355us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:55:18.368138 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.369066 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.370511 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.371515 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.371662 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:18.371872 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 03:55:18.372044 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 03:55:18.372199 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7257,"thread_start_us":7120,"threads_started":1}
I20260808 03:55:18.376051 18853 tablet_service.cc:1467] Processing CreateTablet for tablet d74b5f113f374568ac6565a172189b36 (DEFAULT_TABLE table=test-table [id=458992145b6745a695bf663bc3eb8257]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:55:18.377154 18853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d74b5f113f374568ac6565a172189b36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:55:18.384858 18755 heartbeater.cc:503] Master 127.26.146.190:46823 was elected leader, sending a full tablet report...
I20260808 03:55:18.387303 19192 tablet_bootstrap.cc:492] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959: Bootstrap starting.
I20260808 03:55:18.388111 19193 tablet_bootstrap.cc:492] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478: Bootstrap starting.
I20260808 03:55:18.391506 19196 tablet_bootstrap.cc:492] T d74b5f113f374568ac6565a172189b36 P 442d2f461aa441ff8d4ce07f5bc7dea7: Bootstrap starting.
I20260808 03:55:18.392115 19192 tablet_bootstrap.cc:654] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959: Neither blocks nor log segments found. Creating new log.
I20260808 03:55:18.392431 19193 tablet_bootstrap.cc:654] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478: Neither blocks nor log segments found. Creating new log.
I20260808 03:55:18.393456 19192 log.cc:824] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959: Log is configured to *not* fsync() on all Append() calls
I20260808 03:55:18.393507 19193 log.cc:824] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478: Log is configured to *not* fsync() on all Append() calls
I20260808 03:55:18.394740 19196 tablet_bootstrap.cc:654] T d74b5f113f374568ac6565a172189b36 P 442d2f461aa441ff8d4ce07f5bc7dea7: Neither blocks nor log segments found. Creating new log.
I20260808 03:55:18.395607 19196 log.cc:824] T d74b5f113f374568ac6565a172189b36 P 442d2f461aa441ff8d4ce07f5bc7dea7: Log is configured to *not* fsync() on all Append() calls
I20260808 03:55:18.396474 19193 tablet_bootstrap.cc:492] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478: No bootstrap required, opened a new log
I20260808 03:55:18.396762 19193 ts_tablet_manager.cc:1397] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260808 03:55:18.397701 19192 tablet_bootstrap.cc:492] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959: No bootstrap required, opened a new log
I20260808 03:55:18.397987 19196 tablet_bootstrap.cc:492] T d74b5f113f374568ac6565a172189b36 P 442d2f461aa441ff8d4ce07f5bc7dea7: No bootstrap required, opened a new log
I20260808 03:55:18.397984 19192 ts_tablet_manager.cc:1397] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260808 03:55:18.398211 19196 ts_tablet_manager.cc:1397] T d74b5f113f374568ac6565a172189b36 P 442d2f461aa441ff8d4ce07f5bc7dea7: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 03:55:18.405275 19196 raft_consensus.cc:348] T d74b5f113f374568ac6565a172189b36 P 442d2f461aa441ff8d4ce07f5bc7dea7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } }
I20260808 03:55:18.405692 19196 raft_consensus.cc:374] T d74b5f113f374568ac6565a172189b36 P 442d2f461aa441ff8d4ce07f5bc7dea7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:55:18.405838 19196 raft_consensus.cc:729] T d74b5f113f374568ac6565a172189b36 P 442d2f461aa441ff8d4ce07f5bc7dea7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442d2f461aa441ff8d4ce07f5bc7dea7, State: Initialized, Role: FOLLOWER
I20260808 03:55:18.406406 19196 consensus_queue.cc:260] T d74b5f113f374568ac6565a172189b36 P 442d2f461aa441ff8d4ce07f5bc7dea7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } }
I20260808 03:55:18.407648 18958 heartbeater.cc:503] Master 127.26.146.190:46823 was elected leader, sending a full tablet report...
I20260808 03:55:18.407413 19193 raft_consensus.cc:348] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } }
I20260808 03:55:18.407846 19193 raft_consensus.cc:374] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:55:18.407946 19193 raft_consensus.cc:729] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9ca6c74940547508a2f99ef79259478, State: Initialized, Role: FOLLOWER
I20260808 03:55:18.408052 19192 raft_consensus.cc:348] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } }
I20260808 03:55:18.408350 19196 ts_tablet_manager.cc:1428] T d74b5f113f374568ac6565a172189b36 P 442d2f461aa441ff8d4ce07f5bc7dea7: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260808 03:55:18.408455 19192 raft_consensus.cc:374] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:55:18.408555 19192 raft_consensus.cc:729] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed9d286f2aae46f58fb3a8143e90c959, State: Initialized, Role: FOLLOWER
I20260808 03:55:18.408608 19193 consensus_queue.cc:260] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } }
I20260808 03:55:18.409219 19192 consensus_queue.cc:260] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } }
I20260808 03:55:18.410280 19193 ts_tablet_manager.cc:1428] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478: Time spent starting tablet: real 0.013s user 0.008s sys 0.005s
I20260808 03:55:18.410717 19170 heartbeater.cc:503] Master 127.26.146.190:46823 was elected leader, sending a full tablet report...
I20260808 03:55:18.410835 19192 ts_tablet_manager.cc:1428] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260808 03:55:18.415633 19202 raft_consensus.cc:482] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:55:18.415874 19202 raft_consensus.cc:504] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } }
I20260808 03:55:18.417416 19202 leader_election.cc:288] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed9d286f2aae46f58fb3a8143e90c959 (127.26.146.129:32855), 442d2f461aa441ff8d4ce07f5bc7dea7 (127.26.146.130:35807)
I20260808 03:55:18.423205 19174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.417569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:32855 (local address 127.26.146.131:40241)
0808 03:55:18.417764 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:18.417784 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:55:18.417908 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:55:18.418364 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:55:18.418370 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:55:18.418402 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:55:18.418625 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:55:18.418637 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.419793 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.419796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:55:18.421562 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:55:18.421581 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.422678 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.422683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:55:18.422816 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:18.422840 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 03:55:18.422910 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 03:55:18.422999 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":22}
I20260808 03:55:18.423309 19187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.417748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:40241 (local address 127.26.146.129:32855)
0808 03:55:18.418042 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0808 03:55:18.418048 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:55:18.418067 (+ 19us) server_negotiation.cc:407] Connection header received
0808 03:55:18.418131 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:55:18.418148 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:55:18.418224 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:55:18.418316 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:55:18.418763 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.419676 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.421718 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.422549 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.422697 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:18.422772 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 03:55:18.422945 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 03:55:18.423066 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260808 03:55:18.424253 19189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.418103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:60301 (local address 127.26.146.130:35807)
0808 03:55:18.418256 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 03:55:18.418260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:55:18.418876 (+ 616us) server_negotiation.cc:407] Connection header received
0808 03:55:18.418983 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:55:18.418987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:55:18.419054 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:55:18.419142 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:55:18.419566 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.420357 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.422372 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.423483 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.423624 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:18.423701 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 03:55:18.423984 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 03:55:18.424073 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 03:55:18.424444 19205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.417866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:35807 (local address 127.26.146.131:60301)
0808 03:55:18.418782 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:18.418797 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 03:55:18.418889 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:55:18.419146 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:55:18.419149 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:55:18.419164 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:55:18.419444 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:55:18.419452 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.420486 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.420492 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:55:18.422233 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:55:18.422245 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.423651 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.423656 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:55:18.423835 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:18.423868 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 03:55:18.423965 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 03:55:18.424085 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"mutex_wait_us":36,"thread_start_us":564,"threads_started":1}
I20260808 03:55:18.424276 18685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d74b5f113f374568ac6565a172189b36" candidate_uuid: "f9ca6c74940547508a2f99ef79259478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed9d286f2aae46f58fb3a8143e90c959" is_pre_election: true
I20260808 03:55:18.424821 18685 raft_consensus.cc:2454] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9ca6c74940547508a2f99ef79259478 in term 0.
I20260808 03:55:18.425316 18882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d74b5f113f374568ac6565a172189b36" candidate_uuid: "f9ca6c74940547508a2f99ef79259478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" is_pre_election: true
I20260808 03:55:18.425463 18989 leader_election.cc:302] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed9d286f2aae46f58fb3a8143e90c959, f9ca6c74940547508a2f99ef79259478; no voters:
I20260808 03:55:18.425721 18882 raft_consensus.cc:2454] T d74b5f113f374568ac6565a172189b36 P 442d2f461aa441ff8d4ce07f5bc7dea7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9ca6c74940547508a2f99ef79259478 in term 0.
I20260808 03:55:18.425829 19202 raft_consensus.cc:2781] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:55:18.425964 19202 raft_consensus.cc:482] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:55:18.426067 19202 raft_consensus.cc:3037] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:55:18.428574 19202 raft_consensus.cc:504] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } }
I20260808 03:55:18.429160 19202 leader_election.cc:288] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478 [CANDIDATE]: Term 1 election: Requested vote from peers ed9d286f2aae46f58fb3a8143e90c959 (127.26.146.129:32855), 442d2f461aa441ff8d4ce07f5bc7dea7 (127.26.146.130:35807)
I20260808 03:55:18.429579 18685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d74b5f113f374568ac6565a172189b36" candidate_uuid: "f9ca6c74940547508a2f99ef79259478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed9d286f2aae46f58fb3a8143e90c959"
I20260808 03:55:18.429633 18882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d74b5f113f374568ac6565a172189b36" candidate_uuid: "f9ca6c74940547508a2f99ef79259478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7"
I20260808 03:55:18.429790 18685 raft_consensus.cc:3037] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:55:18.429819 18882 raft_consensus.cc:3037] T d74b5f113f374568ac6565a172189b36 P 442d2f461aa441ff8d4ce07f5bc7dea7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:55:18.432956 18882 raft_consensus.cc:2454] T d74b5f113f374568ac6565a172189b36 P 442d2f461aa441ff8d4ce07f5bc7dea7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9ca6c74940547508a2f99ef79259478 in term 1.
I20260808 03:55:18.432956 18685 raft_consensus.cc:2454] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9ca6c74940547508a2f99ef79259478 in term 1.
I20260808 03:55:18.433368 18989 leader_election.cc:302] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed9d286f2aae46f58fb3a8143e90c959, f9ca6c74940547508a2f99ef79259478; no voters:
I20260808 03:55:18.433633 19202 raft_consensus.cc:2781] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:55:18.434135 19202 raft_consensus.cc:686] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478 [term 1 LEADER]: Becoming Leader. State: Replica: f9ca6c74940547508a2f99ef79259478, State: Running, Role: LEADER
I20260808 03:55:18.434635 19202 consensus_queue.cc:237] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } }
I20260808 03:55:18.439220 18446 catalog_manager.cc:5495] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9ca6c74940547508a2f99ef79259478 (127.26.146.131). New cstate: current_term: 1 leader_uuid: "f9ca6c74940547508a2f99ef79259478" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } health_report { overall_health: HEALTHY } } }
I20260808 03:55:18.459717 19188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.454383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32832 (local address 127.26.146.131:44625)
0808 03:55:18.454594 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 03:55:18.454600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:55:18.454741 (+ 141us) server_negotiation.cc:407] Connection header received
0808 03:55:18.454956 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:55:18.454960 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:55:18.455061 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:55:18.455189 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:55:18.455561 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.456327 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.457679 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.458093 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.458199 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:18.458686 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:55:18.459171 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:55:18.459310 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 03:55:18.459393 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 03:55:18.459470 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 03:55:18.467478 18685 raft_consensus.cc:1261] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [term 1 FOLLOWER]: Refusing update from remote peer f9ca6c74940547508a2f99ef79259478: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:55:18.467478 18882 raft_consensus.cc:1261] T d74b5f113f374568ac6565a172189b36 P 442d2f461aa441ff8d4ce07f5bc7dea7 [term 1 FOLLOWER]: Refusing update from remote peer f9ca6c74940547508a2f99ef79259478: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:55:18.468436 19206 consensus_queue.cc:1035] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:55:18.468753 19202 consensus_queue.cc:1035] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:55:18.496507 18445 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:55:18.496797 18445 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:55:18.500340 18445 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b: Sending DeleteTablet for 3 replicas of tablet d74b5f113f374568ac6565a172189b36
I20260808 03:55:18.501453 18655 tablet_service.cc:1514] Processing DeleteTablet for tablet d74b5f113f374568ac6565a172189b36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:55:18 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:41720
I20260808 03:55:18.501525 19067 tablet_service.cc:1514] Processing DeleteTablet for tablet d74b5f113f374568ac6565a172189b36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:55:18 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:32830
I20260808 03:55:18.501506 18853 tablet_service.cc:1514] Processing DeleteTablet for tablet d74b5f113f374568ac6565a172189b36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:55:18 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:48470
I20260808 03:55:18.502815 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 18510
I20260808 03:55:18.514694 19228 tablet_replica.cc:330] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478: stopping tablet replica
I20260808 03:55:18.515202 19228 raft_consensus.cc:2229] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478 [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:55:18.516041 19228 raft_consensus.cc:2258] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:55:18.516810 19230 tablet_replica.cc:330] T d74b5f113f374568ac6565a172189b36 P 442d2f461aa441ff8d4ce07f5bc7dea7: stopping tablet replica
I20260808 03:55:18.517195 19230 raft_consensus.cc:2229] T d74b5f113f374568ac6565a172189b36 P 442d2f461aa441ff8d4ce07f5bc7dea7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:55:18.518937 19228 ts_tablet_manager.cc:1905] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:55:18.523226 19187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.504084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41726 (local address 127.26.146.129:32855)
0808 03:55:18.504263 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 03:55:18.504269 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:55:18.504288 (+ 19us) server_negotiation.cc:407] Connection header received
0808 03:55:18.504345 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:55:18.504350 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:55:18.504426 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:55:18.504534 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:55:18.506541 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.507701 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.509577 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.510225 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.510348 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:18.515356 (+ 5008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:55:18.515372 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:55:18.515392 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:55:18.515440 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:55:18.519444 (+ 4004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:18.520321 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:18.520330 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:18.520334 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:18.520497 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:18.520976 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:18.520981 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:18.520984 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:18.521509 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:55:18.521816 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0808 03:55:18.522606 (+ 790us) server_negotiation.cc:299] Negotiation successful
0808 03:55:18.522830 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 03:55:18.524279 19230 raft_consensus.cc:2258] T d74b5f113f374568ac6565a172189b36 P 442d2f461aa441ff8d4ce07f5bc7dea7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:55:18.524801 18748 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:55:18.527052 19228 ts_tablet_manager.cc:1918] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:55:18.527208 19228 log.cc:1196] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/d74b5f113f374568ac6565a172189b36
I20260808 03:55:18.527904 19228 ts_tablet_manager.cc:1939] T d74b5f113f374568ac6565a172189b36 P f9ca6c74940547508a2f99ef79259478: Deleting consensus metadata
I20260808 03:55:18.528939 19230 ts_tablet_manager.cc:1905] T d74b5f113f374568ac6565a172189b36 P 442d2f461aa441ff8d4ce07f5bc7dea7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:55:18.529548 18425 catalog_manager.cc:4841] TS f9ca6c74940547508a2f99ef79259478 (127.26.146.131:44625): tablet d74b5f113f374568ac6565a172189b36 (table test-table [id=458992145b6745a695bf663bc3eb8257]) successfully deleted
I20260808 03:55:18.536166 19230 ts_tablet_manager.cc:1918] T d74b5f113f374568ac6565a172189b36 P 442d2f461aa441ff8d4ce07f5bc7dea7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:55:18.536329 19230 log.cc:1196] T d74b5f113f374568ac6565a172189b36 P 442d2f461aa441ff8d4ce07f5bc7dea7: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/d74b5f113f374568ac6565a172189b36
I20260808 03:55:18.536875 19230 ts_tablet_manager.cc:1939] T d74b5f113f374568ac6565a172189b36 P 442d2f461aa441ff8d4ce07f5bc7dea7: Deleting consensus metadata
I20260808 03:55:18.538713 18423 catalog_manager.cc:4841] TS 442d2f461aa441ff8d4ce07f5bc7dea7 (127.26.146.130:35807): tablet d74b5f113f374568ac6565a172189b36 (table test-table [id=458992145b6745a695bf663bc3eb8257]) successfully deleted
I20260808 03:55:18.619903 19227 tablet_replica.cc:330] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959: stopping tablet replica
I20260808 03:55:18.620124 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 18510
I20260808 03:55:18.643337 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 18764
W20260808 03:55:18.643360 18423 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.146.129:32855 (error 108)
I20260808 03:55:18.645598 19185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.645033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:32855 (local address 127.0.0.1:41730)
0808 03:55:18.645273 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:18.645403 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:32855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260808 03:55:18.645876 18423 catalog_manager.cc:4584] TS ed9d286f2aae46f58fb3a8143e90c959 (127.26.146.129:32855): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d74b5f113f374568ac6565a172189b36: Network error: Client connection negotiation failed: client connection to 127.26.146.129:32855: connect: Connection refused (error 111)
I20260808 03:55:18.657943 19189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.644543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48482 (local address 127.26.146.130:35807)
0808 03:55:18.644728 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 03:55:18.644733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:55:18.644751 (+ 18us) server_negotiation.cc:407] Connection header received
0808 03:55:18.644812 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:55:18.644816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:55:18.644888 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:55:18.644979 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:55:18.646807 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.647549 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.648995 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.649384 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.649496 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:18.652257 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:55:18.652270 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:55:18.652286 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:55:18.652320 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:55:18.655167 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:18.656055 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:18.656062 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:18.656064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:18.656203 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:18.656592 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:18.656595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:18.656597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:18.656970 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:55:18.657202 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 03:55:18.657536 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 03:55:18.657697 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 03:55:18.658624 18951 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:55:18.699636 19185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.699092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:32855 (local address 127.0.0.1:41746)
0808 03:55:18.699348 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:18.699462 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:32855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260808 03:55:18.701004 19185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.700544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:32855 (local address 127.0.0.1:41748)
0808 03:55:18.700722 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:18.700816 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:32855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 03:55:18.746285 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 18764
I20260808 03:55:18.761977 19185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.761428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:32855 (local address 127.0.0.1:41758)
0808 03:55:18.761691 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:18.761806 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:32855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260808 03:55:18.763202 19185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.762752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:32855 (local address 127.0.0.1:41770)
0808 03:55:18.762925 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:18.763017 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:32855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 03:55:18.768597 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 18963
I20260808 03:55:18.783386 19188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.769404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32834 (local address 127.26.146.131:44625)
0808 03:55:18.769597 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 03:55:18.769602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:55:18.769621 (+ 19us) server_negotiation.cc:407] Connection header received
0808 03:55:18.769788 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:55:18.769792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:55:18.769882 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:55:18.769971 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:55:18.771573 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.772587 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.773928 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:18.774411 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:18.774527 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:18.777303 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:55:18.777318 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:55:18.777335 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:55:18.777376 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:55:18.780482 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:18.781234 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:18.781242 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:18.781244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:18.781421 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:18.781805 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:18.781808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:18.781810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:18.782151 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:55:18.782475 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0808 03:55:18.782793 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 03:55:18.783015 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 03:55:18.784163 19164 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:55:18.843693 19185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.843227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:32855 (local address 127.0.0.1:41772)
0808 03:55:18.843420 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:18.843533 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:32855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 03:55:18.844779 19185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:18.844374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:32855 (local address 127.0.0.1:41780)
0808 03:55:18.844565 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:18.844667 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:32855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260808 03:55:18.916234 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 18963
I20260808 03:55:18.942680 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 18389
I20260808 03:55:18.943929 18500 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:55:19.055773 19185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:19.055324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:32855 (local address 127.0.0.1:41790)
0808 03:55:19.055503 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:19.055610 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:32855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 03:55:19.055796 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 18389
I20260808 03:55:39.084216 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:46823
--webserver_interface=127.26.146.190
--webserver_port=44071
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:46823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:55:39.414582 19238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:55:39.414882 19238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:55:39.415017 19238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:55:39.422111 19238 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 03:55:39.422225 19238 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 03:55:39.422282 19238 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 03:55:39.422353 19238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:55:39.422428 19238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:55:39.422523 19238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:55:39.422583 19238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:55:39.422660 19238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:55:39.432549 19238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:46823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:46823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=44071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:55:39.433849 19238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:55:39.435202 19238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:55:39.447135 19243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:55:39.447162 19244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:55:39.447096 19246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:55:39.447834 19238 server_base.cc:1034] running on GCE node
I20260808 03:55:39.448608 19238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:55:39.449945 19238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:55:39.451187 19238 hybrid_clock.cc:648] HybridClock initialized: now 1786161339451122 us; error 36 us; skew 500 ppm
I20260808 03:55:39.451575 19238 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:55:39 dist-test-slave-jlzn krb5kdc[17555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161339, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:55:39.457717 19238 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:55:39.459143 19238 webserver.cc:460] Webserver started at http://127.26.146.190:44071/ using document root <none> and password file <none>
I20260808 03:55:39.459862 19238 fs_manager.cc:359] Metadata directory not provided
I20260808 03:55:39.459976 19238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:55:39.465422 19238 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 03:55:39.469276 19255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:39.470643 19238 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 03:55:39.470816 19238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "829c60c206424008a85d262920ecda5b"
format_stamp: "Formatted at 2026-08-08 03:54:53 on dist-test-slave-jlzn"
server_key: "5fbc48436547da729c026a2cb65b3a28"
server_key_iv: "80f879098004bc0e4de54cf421c8b1b0"
server_key_version: "encryptionkey@0"
I20260808 03:55:39.471902 19238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:55:39.510051 19238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:55:39.514271 19238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:55:39.514564 19238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:55:39.532572 19238 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:46823
I20260808 03:55:39.532583 19332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:46823 every 8 connection(s)
I20260808 03:55:39.534102 19238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:55:39.540908 19333 sys_catalog.cc:263] Verifying existing consensus state
I20260808 03:55:39.542797 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 19238
I20260808 03:55:39.543658 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:32855
--local_ip_for_outbound_sockets=127.26.146.129
--tserver_master_addrs=127.26.146.190:46823
--webserver_port=43807
--webserver_interface=127.26.146.129
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:55:39.544193 19333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b: Bootstrap starting.
I20260808 03:55:39.569116 19333 log.cc:824] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b: Log is configured to *not* fsync() on all Append() calls
I20260808 03:55:39.590724 19333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 03:55:39.591380 19333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b: Bootstrap complete.
I20260808 03:55:39.604372 19333 raft_consensus.cc:348] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829c60c206424008a85d262920ecda5b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 46823 } }
I20260808 03:55:39.605576 19333 raft_consensus.cc:729] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 829c60c206424008a85d262920ecda5b, State: Initialized, Role: FOLLOWER
I20260808 03:55:39.606284 19333 consensus_queue.cc:260] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829c60c206424008a85d262920ecda5b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 46823 } }
I20260808 03:55:39.606557 19333 raft_consensus.cc:388] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:55:39.606690 19333 raft_consensus.cc:482] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:55:39.606878 19333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 2 FOLLOWER]: Advancing to term 3
I20260808 03:55:39.612406 19333 raft_consensus.cc:504] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829c60c206424008a85d262920ecda5b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 46823 } }
I20260808 03:55:39.612973 19333 leader_election.cc:302] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 829c60c206424008a85d262920ecda5b; no voters:
I20260808 03:55:39.613652 19333 leader_election.cc:288] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 03:55:39.613797 19339 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 3 FOLLOWER]: Leader election won for term 3
I20260808 03:55:39.615020 19339 raft_consensus.cc:686] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [term 3 LEADER]: Becoming Leader. State: Replica: 829c60c206424008a85d262920ecda5b, State: Running, Role: LEADER
I20260808 03:55:39.615545 19339 consensus_queue.cc:237] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829c60c206424008a85d262920ecda5b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 46823 } }
I20260808 03:55:39.615744 19333 sys_catalog.cc:564] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:55:39.618090 19341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 829c60c206424008a85d262920ecda5b. Latest consensus state: current_term: 3 leader_uuid: "829c60c206424008a85d262920ecda5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829c60c206424008a85d262920ecda5b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 46823 } } }
I20260808 03:55:39.618028 19340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "829c60c206424008a85d262920ecda5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829c60c206424008a85d262920ecda5b" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 46823 } } }
I20260808 03:55:39.618388 19341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [sys.catalog]: This master's current role is: LEADER
I20260808 03:55:39.618384 19340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b [sys.catalog]: This master's current role is: LEADER
I20260808 03:55:39.619685 19345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:55:39.634984 19345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:55:39.636777 19345 catalog_manager.cc:1259] Loaded cluster ID: e0cd49dfd918475d9fe89200fff654be
I20260808 03:55:39.636880 19345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:55:39.641077 19345 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:55:39.651352 19345 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b: Loaded TSK: 0
I20260808 03:55:39.651557 19345 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 829c60c206424008a85d262920ecda5b: Loaded TSK: 1
F20260808 03:55:39.652462 19345 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fbe40e439ad google::LogMessage::Fail() at ??:0
@ 0x7fbe40e4688a google::LogMessage::SendToLog() at ??:0
@ 0x7fbe40e434de google::LogMessage::Flush() at ??:0
@ 0x7fbe40e44a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fbe44ba77d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fbe44b9673e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fbe568721c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fbe56871783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fbe568fd2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fbe568fd0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fbe569475e9 std::function<>::operator()() at ??:0
@ 0x7fbe56875cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fbe56873c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fbe568fb610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fbe568fb439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fbe43a8f345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fbe43a9291c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fbe43a92799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fbe43a6deaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fbe46d476db start_thread at ??:0
@ 0x7fbe3fcdd61f clone at ??:0
W20260808 03:55:39.968286 19335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:55:39.968572 19335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:55:39.968689 19335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:55:39.975890 19335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:55:39.975997 19335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:55:39.976152 19335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:55:39.985692 19335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:32855
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=43807
--tserver_master_addrs=127.26.146.190:46823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:55:39.986948 19335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:55:39.988276 19335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:55:40.001196 19403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:55:40.001134 19399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:55:40.001227 19400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:55:40.001996 19335 server_base.cc:1034] running on GCE node
I20260808 03:55:40.002631 19335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:55:40.003589 19335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:55:40.004796 19335 hybrid_clock.cc:648] HybridClock initialized: now 1786161340004724 us; error 65 us; skew 500 ppm
I20260808 03:55:40.005116 19335 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:55:40 dist-test-slave-jlzn krb5kdc[17555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161340, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:55:40.013890 19335 init.cc:377] Logged in from keytab as kudu/127.26.146.129@KRBTEST.COM (short username kudu)
I20260808 03:55:40.015391 19335 webserver.cc:460] Webserver started at http://127.26.146.129:43807/ using document root <none> and password file <none>
I20260808 03:55:40.016141 19335 fs_manager.cc:359] Metadata directory not provided
I20260808 03:55:40.016268 19335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:55:40.021596 19335 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 03:55:40.024422 19411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:40.025729 19335 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:55:40.025882 19335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "ed9d286f2aae46f58fb3a8143e90c959"
format_stamp: "Formatted at 2026-08-08 03:54:54 on dist-test-slave-jlzn"
server_key: "e602046b5269788d3859e78ba7ef8a66"
server_key_iv: "6522547b41972a325f83b492eeb9159a"
server_key_version: "encryptionkey@0"
I20260808 03:55:40.026870 19335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:55:40.061364 19335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:55:40.066557 19335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:55:40.068580 19335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:55:40.070091 19335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:55:40.072269 19419 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260808 03:55:40.079778 19335 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 03:55:40.079923 19335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.008s user 0.001s sys 0.000s
I20260808 03:55:40.080055 19335 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 03:55:40.084250 19335 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 03:55:40.084349 19335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.000s sys 0.004s
I20260808 03:55:40.084780 19419 tablet_bootstrap.cc:492] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959: Bootstrap starting.
I20260808 03:55:40.112493 19419 log.cc:824] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959: Log is configured to *not* fsync() on all Append() calls
I20260808 03:55:40.123229 19335 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:32855
I20260808 03:55:40.123258 19586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:32855 every 8 connection(s)
I20260808 03:55:40.124876 19335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:55:40.126173 19419 tablet_bootstrap.cc:492] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 03:55:40.126799 19419 tablet_bootstrap.cc:492] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959: Bootstrap complete.
I20260808 03:55:40.128062 19419 ts_tablet_manager.cc:1397] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959: Time spent bootstrapping tablet: real 0.044s user 0.021s sys 0.021s
I20260808 03:55:40.130100 19591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:40.127627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.129:48493)
0808 03:55:40.128515 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:40.128832 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":388,"threads_started":1}
W20260808 03:55:40.130781 19588 heartbeater.cc:644] Failed to heartbeat to 127.26.146.190:46823 (0 consecutive failures): Network error: Failed to ping master at 127.26.146.190:46823: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
I20260808 03:55:40.131736 19591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:40.131311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.129:59887)
0808 03:55:40.131476 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:40.131595 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 03:55:40.132762 19591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:40.132326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.129:52473)
0808 03:55:40.132502 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:40.132610 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 03:55:40.133023 19588 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 03:55:40.134948 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 19335
I20260808 03:55:40.135757 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:35807
--local_ip_for_outbound_sockets=127.26.146.130
--tserver_master_addrs=127.26.146.190:46823
--webserver_port=41263
--webserver_interface=127.26.146.130
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:55:40.136641 19419 raft_consensus.cc:348] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } }
I20260808 03:55:40.137853 19419 raft_consensus.cc:729] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed9d286f2aae46f58fb3a8143e90c959, State: Initialized, Role: FOLLOWER
I20260808 03:55:40.138490 19419 consensus_queue.cc:260] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } }
I20260808 03:55:40.139943 19591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:40.139585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.129:57263)
0808 03:55:40.139730 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:40.139797 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 03:55:40.140100 19419 ts_tablet_manager.cc:1428] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260808 03:55:40.140913 19591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:40.140609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.129:47347)
0808 03:55:40.140740 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:40.140805 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 03:55:40.463510 19593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:55:40.463779 19593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:55:40.463915 19593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:55:40.470773 19593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:55:40.470868 19593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:55:40.470978 19593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:55:40.480296 19593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:35807
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=41263
--tserver_master_addrs=127.26.146.190:46823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:55:40.481441 19593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:55:40.482661 19593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:55:40.495219 19599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:55:40.495273 19601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:55:40.495489 19604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:55:40.496165 19593 server_base.cc:1034] running on GCE node
I20260808 03:55:40.496796 19593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:55:40.497725 19593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:55:40.498921 19593 hybrid_clock.cc:648] HybridClock initialized: now 1786161340498871 us; error 46 us; skew 500 ppm
I20260808 03:55:40.499234 19593 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:55:40 dist-test-slave-jlzn krb5kdc[17555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161340, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:55:40.502252 19593 init.cc:377] Logged in from keytab as kudu/127.26.146.130@KRBTEST.COM (short username kudu)
I20260808 03:55:40.503592 19593 webserver.cc:460] Webserver started at http://127.26.146.130:41263/ using document root <none> and password file <none>
I20260808 03:55:40.504312 19593 fs_manager.cc:359] Metadata directory not provided
I20260808 03:55:40.504436 19593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:55:40.510495 19593 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 03:55:40.513139 19612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:40.514474 19593 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:55:40.514634 19593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "442d2f461aa441ff8d4ce07f5bc7dea7"
format_stamp: "Formatted at 2026-08-08 03:54:54 on dist-test-slave-jlzn"
server_key: "a277e5c3d424f1fc7cdc4131f020f6a5"
server_key_iv: "ada5824d684fa48fb50197a1076d8d11"
server_key_version: "encryptionkey@0"
I20260808 03:55:40.515583 19593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:55:40.567620 19593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:55:40.571784 19593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:55:40.572074 19593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:55:40.573112 19593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:55:40.574509 19593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:55:40.574615 19593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:40.574728 19593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:55:40.574798 19593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:40.605135 19593 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:35807
I20260808 03:55:40.605203 19782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:35807 every 8 connection(s)
I20260808 03:55:40.606521 19593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:55:40.611155 19786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:40.609411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.130:50421)
0808 03:55:40.610104 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:40.610351 (+ 247us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":249,"threads_started":1}
W20260808 03:55:40.611951 19783 heartbeater.cc:644] Failed to heartbeat to 127.26.146.190:46823 (0 consecutive failures): Network error: Failed to ping master at 127.26.146.190:46823: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
I20260808 03:55:40.612697 19786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:40.612402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.130:51849)
0808 03:55:40.612529 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:40.612597 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 03:55:40.613420 19786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:40.613146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.130:32855)
0808 03:55:40.613251 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:40.613320 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 03:55:40.613633 19783 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 03:55:40.614982 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 19593
I20260808 03:55:40.615597 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:44625
--local_ip_for_outbound_sockets=127.26.146.131
--tserver_master_addrs=127.26.146.190:46823
--webserver_port=33611
--webserver_interface=127.26.146.131
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:55:40.944669 19787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:55:40.944962 19787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:55:40.945057 19787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:55:40.952154 19787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:55:40.952261 19787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:55:40.952382 19787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:55:40.961882 19787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:44625
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=33611
--tserver_master_addrs=127.26.146.190:46823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:55:40.963122 19787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:55:40.964390 19787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:55:40.976835 19801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:55:40.976876 19800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:55:40.977382 19803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:55:40.977975 19787 server_base.cc:1034] running on GCE node
I20260808 03:55:40.978523 19787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:55:40.979410 19787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:55:40.980600 19787 hybrid_clock.cc:648] HybridClock initialized: now 1786161340980551 us; error 42 us; skew 500 ppm
I20260808 03:55:40.980904 19787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:55:40 dist-test-slave-jlzn krb5kdc[17555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161340, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:55:40.983981 19787 init.cc:377] Logged in from keytab as kudu/127.26.146.131@KRBTEST.COM (short username kudu)
I20260808 03:55:40.985427 19787 webserver.cc:460] Webserver started at http://127.26.146.131:33611/ using document root <none> and password file <none>
I20260808 03:55:40.986102 19787 fs_manager.cc:359] Metadata directory not provided
I20260808 03:55:40.986227 19787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:55:40.991431 19787 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:55:40.994230 19813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:40.995563 19787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 03:55:40.995733 19787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "f9ca6c74940547508a2f99ef79259478"
format_stamp: "Formatted at 2026-08-08 03:54:55 on dist-test-slave-jlzn"
server_key: "5f0443d6454e8ca1e5a13f9330ff49b1"
server_key_iv: "aab9fcd3b548cc654fb603d5ec4686e4"
server_key_version: "encryptionkey@0"
I20260808 03:55:40.996717 19787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:55:41.030174 19787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:55:41.034502 19787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:55:41.034742 19787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:55:41.035745 19787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:55:41.037087 19787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:55:41.037189 19787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:41.037274 19787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:55:41.037329 19787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:41.068181 19787 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:44625
I20260808 03:55:41.068274 19994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:44625 every 8 connection(s)
I20260808 03:55:41.069626 19787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:55:41.073525 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 19787
I20260808 03:55:41.074697 19998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:41.072544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.131:42935)
0808 03:55:41.073391 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:41.073687 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":288,"threads_started":1}
W20260808 03:55:41.075480 19996 heartbeater.cc:644] Failed to heartbeat to 127.26.146.190:46823 (0 consecutive failures): Network error: Failed to ping master at 127.26.146.190:46823: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
I20260808 03:55:41.076355 19998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:41.076046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.131:36963)
0808 03:55:41.076180 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:41.076246 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 03:55:41.077142 19998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:41.076832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.131:34933)
0808 03:55:41.076949 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:41.077036 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 03:55:41.077363 19996 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 03:55:41.142426 20002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:41.141687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.129:58107)
0808 03:55:41.142143 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:41.142231 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":212,"threads_started":1}
I20260808 03:55:41.427276 20003 raft_consensus.cc:482] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:55:41.427575 20003 raft_consensus.cc:504] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } }
I20260808 03:55:41.429250 20003 leader_election.cc:288] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442d2f461aa441ff8d4ce07f5bc7dea7 (127.26.146.130:35807), f9ca6c74940547508a2f99ef79259478 (127.26.146.131:44625)
Aug 08 03:55:41 dist-test-slave-jlzn krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161340, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.130@KRBTEST.COM
Aug 08 03:55:41 dist-test-slave-jlzn krb5kdc[17555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161340, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.131@KRBTEST.COM
I20260808 03:55:41.454365 20002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:41.429465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:35807 (local address 127.26.146.129:39691)
0808 03:55:41.429709 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:41.429758 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 03:55:41.430390 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:55:41.434875 (+ 4485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:55:41.434886 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:55:41.435893 (+ 1007us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:55:41.436961 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:55:41.436990 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:55:41.439270 (+ 2280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:55:41.439274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:55:41.441478 (+ 2204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:55:41.441494 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:55:41.442881 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:55:41.442887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:55:41.443224 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:41.444152 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:55:41.444181 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:55:41.447120 (+ 2939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:55:41.450744 (+ 3624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:55:41.450753 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:55:41.450767 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:55:41.451362 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:55:41.451838 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:55:41.451843 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:55:41.451847 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:55:41.452048 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:55:41.452811 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:55:41.452819 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:55:41.453155 (+ 336us) client_negotiation.cc:769] Sending connection context
0808 03:55:41.453400 (+ 245us) client_negotiation.cc:240] Negotiation successful
0808 03:55:41.453914 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":419,"spinlock_wait_cycles":14336}
I20260808 03:55:41.454408 20007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:41.429882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:59279 (local address 127.26.146.131:44625)
0808 03:55:41.431340 (+ 1458us) server_negotiation.cc:206] Beginning negotiation
0808 03:55:41.431357 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 03:55:41.431944 (+ 587us) server_negotiation.cc:407] Connection header received
0808 03:55:41.434021 (+ 2077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:55:41.434030 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:55:41.434363 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:55:41.434710 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:55:41.437191 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:41.439072 (+ 1881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:41.441645 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:41.442650 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:41.442853 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:41.447966 (+ 5113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:55:41.447973 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:55:41.447985 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:55:41.448026 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:55:41.450744 (+ 2718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:41.451512 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:41.451519 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:41.451524 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:41.451684 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:41.452188 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:41.452192 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:41.452194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:41.452674 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:55:41.452851 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 03:55:41.453664 (+ 813us) server_negotiation.cc:299] Negotiation successful
0808 03:55:41.454024 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1252,"thread_start_us":1037,"threads_started":1}
I20260808 03:55:41.454425 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:41.429940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:39691 (local address 127.26.146.130:35807)
0808 03:55:41.431689 (+ 1749us) server_negotiation.cc:206] Beginning negotiation
0808 03:55:41.431703 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 03:55:41.431744 (+ 41us) server_negotiation.cc:407] Connection header received
0808 03:55:41.433826 (+ 2082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:55:41.433840 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:55:41.434340 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:55:41.434710 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:55:41.437191 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:41.439072 (+ 1881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:41.441645 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:41.442650 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:41.442853 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:41.447473 (+ 4620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:55:41.447484 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:55:41.447517 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:55:41.447565 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:55:41.450602 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:41.451518 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:41.451524 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:41.451530 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:41.451701 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:41.452173 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:41.452179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:41.452182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:41.452678 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:55:41.452863 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 03:55:41.453666 (+ 803us) server_negotiation.cc:299] Negotiation successful
0808 03:55:41.453992 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1432,"thread_start_us":1328,"threads_started":1}
I20260808 03:55:41.454466 20004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:41.429491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:44625 (local address 127.26.146.129:59279)
0808 03:55:41.431841 (+ 2350us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:41.431857 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:55:41.431957 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:55:41.434875 (+ 2918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:55:41.434886 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:55:41.435893 (+ 1007us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:55:41.436961 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:55:41.436990 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:55:41.439263 (+ 2273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:55:41.439268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:55:41.441478 (+ 2210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:55:41.441494 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:55:41.442856 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:55:41.442866 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:55:41.443228 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:41.444120 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:55:41.444152 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:55:41.447684 (+ 3532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:55:41.450919 (+ 3235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:55:41.450925 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:55:41.450928 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:55:41.451365 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:55:41.451798 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:55:41.451829 (+ 31us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:55:41.451833 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:55:41.452070 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:55:41.452811 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:55:41.452819 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:55:41.453155 (+ 336us) client_negotiation.cc:769] Sending connection context
0808 03:55:41.453400 (+ 245us) client_negotiation.cc:240] Negotiation successful
0808 03:55:41.454069 (+ 669us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2219,"mutex_wait_us":453,"spinlock_wait_cycles":52224,"thread_start_us":2047,"threads_started":1}
W20260808 03:55:41.457919 19416 leader_election.cc:341] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f9ca6c74940547508a2f99ef79259478 (127.26.146.131:44625): Not found: Tablet not found: d74b5f113f374568ac6565a172189b36
W20260808 03:55:41.458802 19413 leader_election.cc:341] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 442d2f461aa441ff8d4ce07f5bc7dea7 (127.26.146.130:35807): Not found: Tablet not found: d74b5f113f374568ac6565a172189b36
I20260808 03:55:41.458928 19413 leader_election.cc:302] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed9d286f2aae46f58fb3a8143e90c959; no voters: 442d2f461aa441ff8d4ce07f5bc7dea7, f9ca6c74940547508a2f99ef79259478
I20260808 03:55:41.459349 20003 raft_consensus.cc:2726] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 03:55:41.614899 20008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:41.614094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.130:43661)
0808 03:55:41.614518 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:41.614666 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":195,"threads_started":1}
I20260808 03:55:42.078646 20009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:42.077864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.131:48717)
0808 03:55:42.078282 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:42.078385 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":192,"threads_started":1}
I20260808 03:55:42.143956 20010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:42.143261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.129:50301)
0808 03:55:42.143681 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:42.143764 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":204,"threads_started":1}
I20260808 03:55:42.616436 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:42.615709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.130:38703)
0808 03:55:42.616195 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:42.616276 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":285,"threads_started":1}
I20260808 03:55:43.003957 20013 raft_consensus.cc:482] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:55:43.004141 20013 raft_consensus.cc:504] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } }
I20260808 03:55:43.004724 20013 leader_election.cc:288] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442d2f461aa441ff8d4ce07f5bc7dea7 (127.26.146.130:35807), f9ca6c74940547508a2f99ef79259478 (127.26.146.131:44625)
W20260808 03:55:43.005697 19416 leader_election.cc:341] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f9ca6c74940547508a2f99ef79259478 (127.26.146.131:44625): Not found: Tablet not found: d74b5f113f374568ac6565a172189b36
W20260808 03:55:43.005875 19413 leader_election.cc:341] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 442d2f461aa441ff8d4ce07f5bc7dea7 (127.26.146.130:35807): Not found: Tablet not found: d74b5f113f374568ac6565a172189b36
I20260808 03:55:43.005985 19413 leader_election.cc:302] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed9d286f2aae46f58fb3a8143e90c959; no voters: 442d2f461aa441ff8d4ce07f5bc7dea7, f9ca6c74940547508a2f99ef79259478
I20260808 03:55:43.006206 20013 raft_consensus.cc:2726] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 03:55:43.080205 20014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:43.079450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.131:44069)
0808 03:55:43.079918 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:43.080005 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":214,"threads_started":1}
I20260808 03:55:43.145746 20015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:43.144867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.129:60721)
0808 03:55:43.145391 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:43.145538 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":223,"threads_started":1}
I20260808 03:55:43.618043 20020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:43.617240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.130:59239)
0808 03:55:43.617795 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:43.617878 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":218,"threads_started":1}
I20260808 03:55:44.081681 20021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:44.080996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.131:53885)
0808 03:55:44.081431 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:44.081516 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":228,"threads_started":1}
I20260808 03:55:44.147290 20023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:44.146614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.129:38473)
0808 03:55:44.147024 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:44.147116 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":205,"threads_started":1}
I20260808 03:55:44.619696 20024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:44.618952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.130:47231)
0808 03:55:44.619388 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:44.619468 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":208,"threads_started":1}
I20260808 03:55:45.083330 20028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:45.082486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.131:42949)
0808 03:55:45.083059 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:45.083155 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":316,"threads_started":1}
I20260808 03:55:45.149114 20029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:45.148374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.129:51823)
0808 03:55:45.148817 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:45.148916 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":244,"threads_started":1}
I20260808 03:55:45.621405 20030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:45.620653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.130:36619)
0808 03:55:45.621158 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:45.621243 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":235,"threads_started":1}
I20260808 03:55:46.085208 20031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:46.084306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.131:60419)
0808 03:55:46.084953 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:46.085055 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":378,"threads_started":1}
I20260808 03:55:46.150871 20032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:46.149966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.129:60081)
0808 03:55:46.150553 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:46.150676 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":326,"threads_started":1}
I20260808 03:55:46.244917 20035 raft_consensus.cc:482] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:55:46.245110 20035 raft_consensus.cc:504] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } }
I20260808 03:55:46.245786 20035 leader_election.cc:288] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442d2f461aa441ff8d4ce07f5bc7dea7 (127.26.146.130:35807), f9ca6c74940547508a2f99ef79259478 (127.26.146.131:44625)
W20260808 03:55:46.246738 19416 leader_election.cc:341] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f9ca6c74940547508a2f99ef79259478 (127.26.146.131:44625): Not found: Tablet not found: d74b5f113f374568ac6565a172189b36
W20260808 03:55:46.246917 19413 leader_election.cc:341] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 442d2f461aa441ff8d4ce07f5bc7dea7 (127.26.146.130:35807): Not found: Tablet not found: d74b5f113f374568ac6565a172189b36
I20260808 03:55:46.247040 19413 leader_election.cc:302] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed9d286f2aae46f58fb3a8143e90c959; no voters: 442d2f461aa441ff8d4ce07f5bc7dea7, f9ca6c74940547508a2f99ef79259478
I20260808 03:55:46.247298 20035 raft_consensus.cc:2726] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 03:55:46.622960 20036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:46.622244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.130:49533)
0808 03:55:46.622704 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:46.622782 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":251,"threads_started":1}
I20260808 03:55:47.086725 20037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:47.085996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.131:42287)
0808 03:55:47.086449 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:47.086531 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":247,"threads_started":1}
I20260808 03:55:47.152438 20038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:47.151691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.129:54929)
0808 03:55:47.152136 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:47.152227 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":224,"threads_started":1}
I20260808 03:55:47.624696 20040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:47.623864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.130:36221)
0808 03:55:47.624390 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:47.624477 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":242,"threads_started":1}
I20260808 03:55:48.088434 20041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:48.087625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.131:60933)
0808 03:55:48.088152 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:48.088234 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":246,"threads_started":1}
I20260808 03:55:48.154017 20042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:48.153251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.129:57993)
0808 03:55:48.153758 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:48.153850 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":297,"threads_started":1}
I20260808 03:55:48.626269 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:48.625568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.130:54223)
0808 03:55:48.625985 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:48.626076 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":211,"threads_started":1}
I20260808 03:55:49.090037 20045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:49.089299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.131:48491)
0808 03:55:49.089761 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:49.089851 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":260,"threads_started":1}
I20260808 03:55:49.155612 20047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:49.154907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.129:58743)
0808 03:55:49.155341 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:49.155424 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":225,"threads_started":1}
I20260808 03:55:49.627946 20048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:49.627081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.130:39713)
0808 03:55:49.627594 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:49.627710 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":226,"threads_started":1}
I20260808 03:55:50.091605 20049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:50.090845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.131:58391)
0808 03:55:50.091329 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:50.091415 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":234,"threads_started":1}
I20260808 03:55:50.158953 20050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:50.157689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.129:35965)
0808 03:55:50.158692 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:50.158787 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":906,"thread_start_us":795,"threads_started":1}
I20260808 03:55:50.629473 20051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:50.628789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.130:58019)
0808 03:55:50.629207 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:50.629287 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":217,"threads_started":1}
I20260808 03:55:50.703501 20052 raft_consensus.cc:482] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:55:50.703743 20052 raft_consensus.cc:504] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } }
I20260808 03:55:50.704470 20052 leader_election.cc:288] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442d2f461aa441ff8d4ce07f5bc7dea7 (127.26.146.130:35807), f9ca6c74940547508a2f99ef79259478 (127.26.146.131:44625)
W20260808 03:55:50.705358 19416 leader_election.cc:341] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f9ca6c74940547508a2f99ef79259478 (127.26.146.131:44625): Not found: Tablet not found: d74b5f113f374568ac6565a172189b36
W20260808 03:55:50.705551 19413 leader_election.cc:341] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 442d2f461aa441ff8d4ce07f5bc7dea7 (127.26.146.130:35807): Not found: Tablet not found: d74b5f113f374568ac6565a172189b36
I20260808 03:55:50.705677 19413 leader_election.cc:302] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed9d286f2aae46f58fb3a8143e90c959; no voters: 442d2f461aa441ff8d4ce07f5bc7dea7, f9ca6c74940547508a2f99ef79259478
I20260808 03:55:50.705909 20052 raft_consensus.cc:2726] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 03:55:51.093220 20053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:51.092509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.131:43267)
0808 03:55:51.092943 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:51.093044 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":229,"threads_started":1}
I20260808 03:55:51.160725 20054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:51.159884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.129:59475)
0808 03:55:51.160374 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:51.160498 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":234,"threads_started":1}
I20260808 03:55:51.631008 20055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:51.630314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.130:50291)
0808 03:55:51.630733 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:51.630826 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":208,"threads_started":1}
I20260808 03:55:52.094770 20056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:52.094018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.131:48399)
0808 03:55:52.094483 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:52.094609 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":230,"threads_started":1}
I20260808 03:55:52.162351 20057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:52.161622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.129:50351)
0808 03:55:52.162080 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:52.162168 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":229,"threads_started":1}
I20260808 03:55:52.632869 20058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:52.632184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.130:43659)
0808 03:55:52.632612 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:52.632697 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":215,"threads_started":1}
I20260808 03:55:52.913856 20059 raft_consensus.cc:482] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:55:52.914074 20059 raft_consensus.cc:504] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d286f2aae46f58fb3a8143e90c959" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 32855 } } peers { permanent_uuid: "442d2f461aa441ff8d4ce07f5bc7dea7" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 35807 } } peers { permanent_uuid: "f9ca6c74940547508a2f99ef79259478" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 44625 } }
I20260808 03:55:52.914834 20059 leader_election.cc:288] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 442d2f461aa441ff8d4ce07f5bc7dea7 (127.26.146.130:35807), f9ca6c74940547508a2f99ef79259478 (127.26.146.131:44625)
W20260808 03:55:52.915694 19413 leader_election.cc:341] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 442d2f461aa441ff8d4ce07f5bc7dea7 (127.26.146.130:35807): Not found: Tablet not found: d74b5f113f374568ac6565a172189b36
W20260808 03:55:52.915880 19416 leader_election.cc:341] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f9ca6c74940547508a2f99ef79259478 (127.26.146.131:44625): Not found: Tablet not found: d74b5f113f374568ac6565a172189b36
I20260808 03:55:52.916004 19416 leader_election.cc:302] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed9d286f2aae46f58fb3a8143e90c959; no voters: 442d2f461aa441ff8d4ce07f5bc7dea7, f9ca6c74940547508a2f99ef79259478
I20260808 03:55:52.916270 20059 raft_consensus.cc:2726] T d74b5f113f374568ac6565a172189b36 P ed9d286f2aae46f58fb3a8143e90c959 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 03:55:53.096422 20060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:53.095644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.131:57767)
0808 03:55:53.096168 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:53.096251 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":274,"threads_started":1}
I20260808 03:55:53.164139 20062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:53.163254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.129:43847)
0808 03:55:53.163770 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:53.163935 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":273,"threads_started":1}
I20260808 03:55:53.634413 20063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:53.633680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.130:36037)
0808 03:55:53.634121 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:53.634220 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":218,"threads_started":1}
I20260808 03:55:54.098098 20064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:54.097340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.131:52029)
0808 03:55:54.097834 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:54.097925 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":246,"threads_started":1}
I20260808 03:55:54.165702 20065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:54.164992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.129:56095)
0808 03:55:54.165434 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:54.165515 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":227,"threads_started":1}
I20260808 03:55:54.636101 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:54.635328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.130:51515)
0808 03:55:54.635770 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:54.635895 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":246,"threads_started":1}
I20260808 03:55:55.099653 20070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:55.098922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.131:46679)
0808 03:55:55.099359 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:55.099442 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":222,"threads_started":1}
I20260808 03:55:55.167652 20072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:55.166912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.129:49491)
0808 03:55:55.167380 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:55.167470 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":237,"threads_started":1}
I20260808 03:55:55.637888 20074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:55.637171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.130:36367)
0808 03:55:55.637605 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:55.637705 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":212,"threads_started":1}
I20260808 03:55:56.083598 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 19335
I20260808 03:55:56.101509 20075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:56.084682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45212 (local address 127.26.146.129:32855)
0808 03:55:56.085235 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0808 03:55:56.085244 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:55:56.085275 (+ 31us) server_negotiation.cc:407] Connection header received
0808 03:55:56.085348 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:55:56.085355 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:55:56.085650 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:55:56.085818 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:55:56.087548 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:56.088533 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:56.089761 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:56.090323 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:56.090424 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:56.095924 (+ 5500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:55:56.095937 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:55:56.095952 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:55:56.095993 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:55:56.098653 (+ 2660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:56.099286 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:56.099292 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:56.099294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:56.099430 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:56.099791 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:56.099793 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:56.099795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:56.100224 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:55:56.100493 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0808 03:55:56.100852 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 03:55:56.101156 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":690,"threads_started":1}
I20260808 03:55:56.101619 20076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:56.100811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.131:50251)
0808 03:55:56.101292 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:56.101428 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":407,"threads_started":1}
I20260808 03:55:56.102799 19581 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:55:56.191963 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 19335
I20260808 03:55:56.192133 20078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:56.190959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:46823 (local address 127.26.146.129:39407)
0808 03:55:56.191608 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:56.191790 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.190:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":308,"threads_started":1}
I20260808 03:55:56.217178 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 19593
I20260808 03:55:56.230273 20079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:56.217891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46584 (local address 127.26.146.130:35807)
0808 03:55:56.218409 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0808 03:55:56.218414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:55:56.218429 (+ 15us) server_negotiation.cc:407] Connection header received
0808 03:55:56.218483 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:55:56.218487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:55:56.218545 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:55:56.218639 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:55:56.220135 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:56.220918 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:56.222173 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:56.222604 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:56.222717 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:56.225297 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:55:56.225303 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:55:56.225306 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:55:56.225339 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:55:56.227642 (+ 2303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:56.228313 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:56.228318 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:56.228322 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:56.228472 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:56.228854 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:56.228858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:56.228860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:56.229148 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:55:56.229396 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 03:55:56.229707 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 03:55:56.229926 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":293,"threads_started":1}
I20260808 03:55:56.230980 19775 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:55:56.313184 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 19593
I20260808 03:55:56.336925 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 19787
I20260808 03:55:56.350919 20082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:56.337896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43378 (local address 127.26.146.131:44625)
0808 03:55:56.338467 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0808 03:55:56.338472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:55:56.338488 (+ 16us) server_negotiation.cc:407] Connection header received
0808 03:55:56.338543 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:55:56.338556 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:55:56.338615 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:55:56.338729 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:55:56.340215 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:56.341072 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:56.342497 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:56.342949 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:56.343066 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:56.345726 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:55:56.345731 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:55:56.345734 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:55:56.345766 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:55:56.348301 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:56.348974 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:56.348978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:56.348981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:56.349120 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:56.349482 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:56.349486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:56.349489 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:56.349756 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:55:56.350017 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0808 03:55:56.350324 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 03:55:56.350537 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":295,"threads_started":1}
I20260808 03:55:56.351637 19985 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:55:56.433911 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 19787
2026-08-08T03:55:56Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64486 ms)
[----------] 30 tests from SecurityITest (298291 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:55:56 dist-test-slave-jlzn krb5kdc[20086](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:55:56 dist-test-slave-jlzn krb5kdc[20086](info): set up 2 sockets
Aug 08 03:55:56 dist-test-slave-jlzn krb5kdc[20086](info): commencing operation
krb5kdc: starting...
W20260808 03:55:57.649648 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:55:57 dist-test-slave-jlzn krb5kdc[20086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:55:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:55:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:55:57.937616 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:41303
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:39839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:41303
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:55:58.267490 20102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:55:58.267784 20102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:55:58.267927 20102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:55:58.275203 20102 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:55:58.275307 20102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:55:58.275353 20102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:55:58.275414 20102 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:55:58.275458 20102 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:55:58.285472 20102 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:39839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:41303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:41303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:55:58.286720 20102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:55:58.288036 20102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:55:58.299857 20111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:55:58.299896 20112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:55:58.299787 20115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:55:58.300697 20102 server_base.cc:1034] running on GCE node
I20260808 03:55:58.301412 20102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:55:58.302736 20102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:55:58.303907 20102 hybrid_clock.cc:648] HybridClock initialized: now 1786161358303850 us; error 39 us; skew 500 ppm
I20260808 03:55:58.304214 20102 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:55:58 dist-test-slave-jlzn krb5kdc[20086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161358, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:55:58.307353 20102 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:55:58.308880 20102 webserver.cc:460] Webserver started at http://127.26.146.190:38299/ using document root <none> and password file <none>
I20260808 03:55:58.309562 20102 fs_manager.cc:359] Metadata directory not provided
I20260808 03:55:58.309680 20102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:55:58.309974 20102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:55:58.312966 20102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "e134eed8ab7047a6884f66d8464e0fb5"
format_stamp: "Formatted at 2026-08-08 03:55:58 on dist-test-slave-jlzn"
server_key: "9840f8ddf638aa260fe629039996309f"
server_key_iv: "e0d149b47509b341b8a3a455797acf38"
server_key_version: "encryptionkey@0"
I20260808 03:55:58.313702 20102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "e134eed8ab7047a6884f66d8464e0fb5"
format_stamp: "Formatted at 2026-08-08 03:55:58 on dist-test-slave-jlzn"
server_key: "9840f8ddf638aa260fe629039996309f"
server_key_iv: "e0d149b47509b341b8a3a455797acf38"
server_key_version: "encryptionkey@0"
I20260808 03:55:58.318761 20102 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 03:55:58.322100 20127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:58.323356 20102 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260808 03:55:58.323504 20102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "e134eed8ab7047a6884f66d8464e0fb5"
format_stamp: "Formatted at 2026-08-08 03:55:58 on dist-test-slave-jlzn"
server_key: "9840f8ddf638aa260fe629039996309f"
server_key_iv: "e0d149b47509b341b8a3a455797acf38"
server_key_version: "encryptionkey@0"
I20260808 03:55:58.323683 20102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:55:58.344558 20102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:55:58.348321 20102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:55:58.348572 20102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:55:58.365890 20102 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:41303
I20260808 03:55:58.365926 20201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:41303 every 8 connection(s)
I20260808 03:55:58.367168 20102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:55:58.370865 20202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:55:58.374713 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 20102
I20260808 03:55:58.374938 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:55:58.375772 27210 external_mini_cluster.cc:1428] Setting key b26ad2f7dc12800c25cc0329b3bc1ab5
I20260808 03:55:58.384191 20202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e134eed8ab7047a6884f66d8464e0fb5: Bootstrap starting.
I20260808 03:55:58.387840 20202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e134eed8ab7047a6884f66d8464e0fb5: Neither blocks nor log segments found. Creating new log.
I20260808 03:55:58.389142 20202 log.cc:824] T 00000000000000000000000000000000 P e134eed8ab7047a6884f66d8464e0fb5: Log is configured to *not* fsync() on all Append() calls
Aug 08 03:55:58 dist-test-slave-jlzn krb5kdc[20086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:55:58.392637 20202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e134eed8ab7047a6884f66d8464e0fb5: No bootstrap required, opened a new log
I20260808 03:55:58.400379 20205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:58.377797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55824 (local address 127.26.146.190:41303)
0808 03:55:58.378513 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0808 03:55:58.378524 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:55:58.378569 (+ 45us) server_negotiation.cc:407] Connection header received
0808 03:55:58.380221 (+ 1652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:55:58.380244 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:55:58.380621 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:55:58.380931 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:55:58.382737 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:58.384129 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:58.385963 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:58.386705 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:58.386831 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:58.391999 (+ 5168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:55:58.392016 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:55:58.392035 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:55:58.392087 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:55:58.395966 (+ 3879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:58.396601 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:58.396609 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:58.396617 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:58.396826 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:58.397178 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:58.397184 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:58.397187 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:58.397867 (+ 680us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:55:58.398165 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0808 03:55:58.398575 (+ 410us) server_negotiation.cc:299] Negotiation successful
0808 03:55:58.399016 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":330,"threads_started":1}
I20260808 03:55:58.402913 20202 raft_consensus.cc:348] T 00000000000000000000000000000000 P e134eed8ab7047a6884f66d8464e0fb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e134eed8ab7047a6884f66d8464e0fb5" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 41303 } }
I20260808 03:55:58.403329 20202 raft_consensus.cc:374] T 00000000000000000000000000000000 P e134eed8ab7047a6884f66d8464e0fb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:55:58.403450 20202 raft_consensus.cc:729] T 00000000000000000000000000000000 P e134eed8ab7047a6884f66d8464e0fb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e134eed8ab7047a6884f66d8464e0fb5, State: Initialized, Role: FOLLOWER
I20260808 03:55:58.404031 20202 consensus_queue.cc:260] T 00000000000000000000000000000000 P e134eed8ab7047a6884f66d8464e0fb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e134eed8ab7047a6884f66d8464e0fb5" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 41303 } }
I20260808 03:55:58.404248 20202 raft_consensus.cc:388] T 00000000000000000000000000000000 P e134eed8ab7047a6884f66d8464e0fb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:55:58.404373 20202 raft_consensus.cc:482] T 00000000000000000000000000000000 P e134eed8ab7047a6884f66d8464e0fb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:55:58.404521 20202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e134eed8ab7047a6884f66d8464e0fb5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:55:58.406816 20202 raft_consensus.cc:504] T 00000000000000000000000000000000 P e134eed8ab7047a6884f66d8464e0fb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e134eed8ab7047a6884f66d8464e0fb5" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 41303 } }
I20260808 03:55:58.407302 20202 leader_election.cc:302] T 00000000000000000000000000000000 P e134eed8ab7047a6884f66d8464e0fb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e134eed8ab7047a6884f66d8464e0fb5; no voters:
I20260808 03:55:58.409302 20202 leader_election.cc:288] T 00000000000000000000000000000000 P e134eed8ab7047a6884f66d8464e0fb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:55:58.409459 20207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e134eed8ab7047a6884f66d8464e0fb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:55:58.410157 20207 raft_consensus.cc:686] T 00000000000000000000000000000000 P e134eed8ab7047a6884f66d8464e0fb5 [term 1 LEADER]: Becoming Leader. State: Replica: e134eed8ab7047a6884f66d8464e0fb5, State: Running, Role: LEADER
I20260808 03:55:58.410676 20207 consensus_queue.cc:237] T 00000000000000000000000000000000 P e134eed8ab7047a6884f66d8464e0fb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e134eed8ab7047a6884f66d8464e0fb5" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 41303 } }
I20260808 03:55:58.411300 20202 sys_catalog.cc:564] T 00000000000000000000000000000000 P e134eed8ab7047a6884f66d8464e0fb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:55:58.413290 20209 sys_catalog.cc:455] T 00000000000000000000000000000000 P e134eed8ab7047a6884f66d8464e0fb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e134eed8ab7047a6884f66d8464e0fb5. Latest consensus state: current_term: 1 leader_uuid: "e134eed8ab7047a6884f66d8464e0fb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e134eed8ab7047a6884f66d8464e0fb5" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 41303 } } }
I20260808 03:55:58.413733 20209 sys_catalog.cc:458] T 00000000000000000000000000000000 P e134eed8ab7047a6884f66d8464e0fb5 [sys.catalog]: This master's current role is: LEADER
I20260808 03:55:58.414311 20208 sys_catalog.cc:455] T 00000000000000000000000000000000 P e134eed8ab7047a6884f66d8464e0fb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e134eed8ab7047a6884f66d8464e0fb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e134eed8ab7047a6884f66d8464e0fb5" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 41303 } } }
I20260808 03:55:58.414602 20208 sys_catalog.cc:458] T 00000000000000000000000000000000 P e134eed8ab7047a6884f66d8464e0fb5 [sys.catalog]: This master's current role is: LEADER
I20260808 03:55:58.414597 20224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:55:58.419896 20224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:55:58.428990 20224 catalog_manager.cc:1347] Generated new cluster ID: 60c8bfadee0d40d7ad71c5984b1febcd
I20260808 03:55:58.429086 20224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:55:58.447388 20224 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:55:58.448643 20224 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:55:58.458577 20224 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e134eed8ab7047a6884f66d8464e0fb5: Generated new TSK 0
I20260808 03:55:58.459321 20224 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
I20260808 03:55:58.538794 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:41303
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:39839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:55:58.871757 20244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:55:58.872223 20244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:55:58.872386 20244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:55:58.880808 20244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:55:58.880959 20244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:55:58.881155 20244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:55:58.895524 20244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:39839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:41303
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:55:58.897157 20244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:55:58.898861 20244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:55:58.912077 20250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:55:58.912098 20251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:55:58.912099 20253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:55:58.912883 20244 server_base.cc:1034] running on GCE node
I20260808 03:55:58.913447 20244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:55:58.914335 20244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:55:58.915585 20244 hybrid_clock.cc:648] HybridClock initialized: now 1786161358915537 us; error 42 us; skew 500 ppm
I20260808 03:55:58.915928 20244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:55:58 dist-test-slave-jlzn krb5kdc[20086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161358, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:55:58.918897 20244 init.cc:377] Logged in from keytab as kudu/127.26.146.129@KRBTEST.COM (short username kudu)
I20260808 03:55:58.920379 20244 webserver.cc:460] Webserver started at http://127.26.146.129:40117/ using document root <none> and password file <none>
I20260808 03:55:58.921013 20244 fs_manager.cc:359] Metadata directory not provided
I20260808 03:55:58.921125 20244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:55:58.921418 20244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:55:58.924294 20244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "b9c3fc99b06b461c91f1d24d9c29e1e3"
format_stamp: "Formatted at 2026-08-08 03:55:58 on dist-test-slave-jlzn"
server_key: "0c85ba04db59e1218fd6e64f01ca82cf"
server_key_iv: "a12ed8726dff880d340e682013a95329"
server_key_version: "encryptionkey@0"
I20260808 03:55:58.925036 20244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "b9c3fc99b06b461c91f1d24d9c29e1e3"
format_stamp: "Formatted at 2026-08-08 03:55:58 on dist-test-slave-jlzn"
server_key: "0c85ba04db59e1218fd6e64f01ca82cf"
server_key_iv: "a12ed8726dff880d340e682013a95329"
server_key_version: "encryptionkey@0"
I20260808 03:55:58.930258 20244 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:55:58.933669 20266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:58.934947 20244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 03:55:58.935093 20244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "b9c3fc99b06b461c91f1d24d9c29e1e3"
format_stamp: "Formatted at 2026-08-08 03:55:58 on dist-test-slave-jlzn"
server_key: "0c85ba04db59e1218fd6e64f01ca82cf"
server_key_iv: "a12ed8726dff880d340e682013a95329"
server_key_version: "encryptionkey@0"
I20260808 03:55:58.935277 20244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:55:58.951103 20244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:55:58.955020 20244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:55:58.955281 20244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:55:58.956254 20244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:55:58.957607 20244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:55:58.957679 20244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:58.957774 20244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:55:58.957836 20244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:58.988488 20244 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:33895
I20260808 03:55:58.988536 20434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:33895 every 8 connection(s)
I20260808 03:55:58.989732 20244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:55:58.997677 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 20244
I20260808 03:55:58.997898 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:55:58.998615 27210 external_mini_cluster.cc:1428] Setting key 26af902ef173cb0ba5fccc652be0a8e5
Aug 08 03:55:58 dist-test-slave-jlzn krb5kdc[20086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161358, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:55:59.009421 20438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:58.992208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:42605 (local address 127.26.146.190:41303)
0808 03:55:58.992641 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0808 03:55:58.992648 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:55:58.993619 (+ 971us) server_negotiation.cc:407] Connection header received
0808 03:55:58.995427 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:55:58.995431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:55:58.995499 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:55:58.995596 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:55:58.997074 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:58.998175 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:58.999869 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:59.000488 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:59.000605 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:59.003944 (+ 3339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:55:59.003949 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:55:59.003952 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:55:59.003984 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:55:59.006441 (+ 2457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:59.007136 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:59.007142 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:59.007145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:59.007268 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:59.007691 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:59.007696 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:59.007700 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:59.008001 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:55:59.008148 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 03:55:59.008840 (+ 692us) server_negotiation.cc:299] Negotiation successful
0808 03:55:59.009076 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":192,"threads_started":1}
I20260808 03:55:59.010380 20439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:58.992754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:41303 (local address 127.26.146.129:42605)
0808 03:55:58.993461 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:58.993498 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 03:55:58.995152 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:55:58.995780 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:55:58.995787 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:55:58.996355 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:55:58.996909 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:55:58.996927 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:55:58.998319 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:55:58.998323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:55:58.999706 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:55:58.999719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:55:59.000713 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:55:59.000721 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:55:59.000954 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:59.001574 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:55:59.001599 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:55:59.003721 (+ 2122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:55:59.006578 (+ 2857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:55:59.006587 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:55:59.006603 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:55:59.007019 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:55:59.007373 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:55:59.007378 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:55:59.007382 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:55:59.007584 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:55:59.008133 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:55:59.008141 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:55:59.008463 (+ 322us) client_negotiation.cc:769] Sending connection context
0808 03:55:59.008729 (+ 266us) client_negotiation.cc:240] Negotiation successful
0808 03:55:59.009254 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":279,"threads_started":1}
I20260808 03:55:59.012028 20436 heartbeater.cc:347] Connected to a master server at 127.26.146.190:41303
I20260808 03:55:59.012382 20436 heartbeater.cc:464] Registering TS with master...
I20260808 03:55:59.013119 20436 heartbeater.cc:511] Master 127.26.146.190:41303 requested a full tablet report, sending...
I20260808 03:55:59.015223 20151 ts_manager.cc:194] Registered new tserver with Master: b9c3fc99b06b461c91f1d24d9c29e1e3 (127.26.146.129:33895)
I20260808 03:55:59.016696 20151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.129@KRBTEST.COM'} at 127.26.146.129:42605
WARNING: no policy specified for kudu/127.26.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
I20260808 03:55:59.058602 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:41303
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:39839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:55:59.384389 20445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:55:59.384665 20445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:55:59.384795 20445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:55:59.391939 20445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:55:59.392043 20445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:55:59.392194 20445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:55:59.401648 20445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:39839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:41303
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:55:59.402856 20445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:55:59.404194 20445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:55:59.416915 20452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:55:59.416987 20453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:55:59.417137 20455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:55:59.417900 20445 server_base.cc:1034] running on GCE node
I20260808 03:55:59.418427 20445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:55:59.419251 20445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:55:59.420444 20445 hybrid_clock.cc:648] HybridClock initialized: now 1786161359420400 us; error 32 us; skew 500 ppm
I20260808 03:55:59.420745 20445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:55:59 dist-test-slave-jlzn krb5kdc[20086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161359, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:55:59.423779 20445 init.cc:377] Logged in from keytab as kudu/127.26.146.130@KRBTEST.COM (short username kudu)
I20260808 03:55:59.425230 20445 webserver.cc:460] Webserver started at http://127.26.146.130:36967/ using document root <none> and password file <none>
I20260808 03:55:59.425858 20445 fs_manager.cc:359] Metadata directory not provided
I20260808 03:55:59.425966 20445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:55:59.426265 20445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:55:59.429117 20445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "e2ddb0f966014a6e95e87d3be16ee6ef"
format_stamp: "Formatted at 2026-08-08 03:55:59 on dist-test-slave-jlzn"
server_key: "270218cf9f0c47f1f141c9a477c811c3"
server_key_iv: "33ee93ed9ddb3788cf84c278878851e0"
server_key_version: "encryptionkey@0"
I20260808 03:55:59.429842 20445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "e2ddb0f966014a6e95e87d3be16ee6ef"
format_stamp: "Formatted at 2026-08-08 03:55:59 on dist-test-slave-jlzn"
server_key: "270218cf9f0c47f1f141c9a477c811c3"
server_key_iv: "33ee93ed9ddb3788cf84c278878851e0"
server_key_version: "encryptionkey@0"
I20260808 03:55:59.434756 20445 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 03:55:59.438097 20464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:59.439389 20445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:55:59.439554 20445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "e2ddb0f966014a6e95e87d3be16ee6ef"
format_stamp: "Formatted at 2026-08-08 03:55:59 on dist-test-slave-jlzn"
server_key: "270218cf9f0c47f1f141c9a477c811c3"
server_key_iv: "33ee93ed9ddb3788cf84c278878851e0"
server_key_version: "encryptionkey@0"
I20260808 03:55:59.439742 20445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:55:59.457024 20445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:55:59.460965 20445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:55:59.461233 20445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:55:59.462163 20445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:55:59.463474 20445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:55:59.463553 20445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:59.463675 20445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:55:59.463727 20445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:59.494606 20445 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:36575
I20260808 03:55:59.494686 20616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:36575 every 8 connection(s)
I20260808 03:55:59.495906 20445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:55:59.505479 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 20445
I20260808 03:55:59.505686 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:55:59.506229 27210 external_mini_cluster.cc:1428] Setting key 0d2832e5b5266ddbdb6be38e5de23be9
Aug 08 03:55:59 dist-test-slave-jlzn krb5kdc[20086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161359, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:55:59.516201 20438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:59.498518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:34989 (local address 127.26.146.190:41303)
0808 03:55:59.498718 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 03:55:59.498723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:55:59.499869 (+ 1146us) server_negotiation.cc:407] Connection header received
0808 03:55:59.501716 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:55:59.501722 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:55:59.501802 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:55:59.501890 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:55:59.503438 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:59.504219 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:59.506074 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:55:59.506549 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:55:59.506659 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:59.510585 (+ 3926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:55:59.510590 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:55:59.510599 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:55:59.510632 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:55:59.513215 (+ 2583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:59.514018 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:59.514024 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:59.514026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:59.514160 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:55:59.514588 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:55:59.514591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:55:59.514599 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:55:59.514874 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:55:59.514992 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 03:55:59.515732 (+ 740us) server_negotiation.cc:299] Negotiation successful
0808 03:55:59.515946 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260808 03:55:59.517518 20619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:55:59.498924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:41303 (local address 127.26.146.130:34989)
0808 03:55:59.499648 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0808 03:55:59.499687 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 03:55:59.501448 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:55:59.502144 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:55:59.502153 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:55:59.502706 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:55:59.503263 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:55:59.503281 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:55:59.504370 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:55:59.504375 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:55:59.505923 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:55:59.505939 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:55:59.506791 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:55:59.506798 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:55:59.507029 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:55:59.507708 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:55:59.507735 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:55:59.510361 (+ 2626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:55:59.513346 (+ 2985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:55:59.513353 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:55:59.513369 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:55:59.513889 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:55:59.514262 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:55:59.514266 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:55:59.514269 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:55:59.514497 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:55:59.514993 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:55:59.515000 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:55:59.515396 (+ 396us) client_negotiation.cc:769] Sending connection context
0808 03:55:59.515681 (+ 285us) client_negotiation.cc:240] Negotiation successful
0808 03:55:59.516236 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":286,"threads_started":1}
I20260808 03:55:59.518857 20617 heartbeater.cc:347] Connected to a master server at 127.26.146.190:41303
I20260808 03:55:59.519176 20617 heartbeater.cc:464] Registering TS with master...
I20260808 03:55:59.520009 20617 heartbeater.cc:511] Master 127.26.146.190:41303 requested a full tablet report, sending...
I20260808 03:55:59.521565 20151 ts_manager.cc:194] Registered new tserver with Master: e2ddb0f966014a6e95e87d3be16ee6ef (127.26.146.130:36575)
I20260808 03:55:59.522635 20151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.130@KRBTEST.COM'} at 127.26.146.130:34989
WARNING: no policy specified for kudu/127.26.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
I20260808 03:55:59.569033 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:41303
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:39839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:55:59.891184 20624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:55:59.891448 20624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:55:59.891568 20624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:55:59.898720 20624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:55:59.898826 20624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:55:59.898945 20624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:55:59.908440 20624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:39839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:41303
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:55:59.909605 20624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:55:59.910842 20624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:55:59.923696 20632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:55:59.923736 20633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:55:59.923712 20635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:55:59.924748 20624 server_base.cc:1034] running on GCE node
I20260808 03:55:59.925343 20624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:55:59.926235 20624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:55:59.927428 20624 hybrid_clock.cc:648] HybridClock initialized: now 1786161359927387 us; error 34 us; skew 500 ppm
I20260808 03:55:59.927717 20624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:55:59 dist-test-slave-jlzn krb5kdc[20086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161359, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:55:59.930897 20624 init.cc:377] Logged in from keytab as kudu/127.26.146.131@KRBTEST.COM (short username kudu)
I20260808 03:55:59.932367 20624 webserver.cc:460] Webserver started at http://127.26.146.131:42239/ using document root <none> and password file <none>
I20260808 03:55:59.933002 20624 fs_manager.cc:359] Metadata directory not provided
I20260808 03:55:59.933109 20624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:55:59.933373 20624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:55:59.936218 20624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "54193318ba2f4c53b302457b4a82fdeb"
format_stamp: "Formatted at 2026-08-08 03:55:59 on dist-test-slave-jlzn"
server_key: "e88fc441bf33ede2669d35328f4a5f3c"
server_key_iv: "bf01adc4f26099f060f4695c3c777760"
server_key_version: "encryptionkey@0"
I20260808 03:55:59.936923 20624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "54193318ba2f4c53b302457b4a82fdeb"
format_stamp: "Formatted at 2026-08-08 03:55:59 on dist-test-slave-jlzn"
server_key: "e88fc441bf33ede2669d35328f4a5f3c"
server_key_iv: "bf01adc4f26099f060f4695c3c777760"
server_key_version: "encryptionkey@0"
I20260808 03:55:59.941982 20624 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:55:59.945343 20645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:59.946596 20624 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 03:55:59.946736 20624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "54193318ba2f4c53b302457b4a82fdeb"
format_stamp: "Formatted at 2026-08-08 03:55:59 on dist-test-slave-jlzn"
server_key: "e88fc441bf33ede2669d35328f4a5f3c"
server_key_iv: "bf01adc4f26099f060f4695c3c777760"
server_key_version: "encryptionkey@0"
I20260808 03:55:59.946903 20624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:55:59.969470 20624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:55:59.973446 20624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:55:59.973721 20624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:55:59.974645 20624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:55:59.975939 20624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:55:59.976012 20624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:55:59.976122 20624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:55:59.976172 20624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:00.007947 20624 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:37119
I20260808 03:56:00.008013 20827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:37119 every 8 connection(s)
I20260808 03:56:00.009204 20624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:56:00.015713 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 20624
I20260808 03:56:00.015942 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:56:00.016533 27210 external_mini_cluster.cc:1428] Setting key c2a5ee6b9519c7c84cb71f18a5607516
I20260808 03:56:00.019411 20436 heartbeater.cc:503] Master 127.26.146.190:41303 was elected leader, sending a full tablet report...
Aug 08 03:56:00 dist-test-slave-jlzn krb5kdc[20086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161359, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:56:00.029742 20438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:00.011728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:53209 (local address 127.26.146.190:41303)
0808 03:56:00.011923 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:00.011927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:00.013134 (+ 1207us) server_negotiation.cc:407] Connection header received
0808 03:56:00.015090 (+ 1956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:00.015095 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:00.015183 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:00.015272 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:00.017316 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.018325 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.020312 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.020881 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.021013 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:00.024339 (+ 3326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:00.024343 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:00.024346 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:00.024377 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:00.026749 (+ 2372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:00.027475 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:00.027479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:00.027482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:00.027618 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:00.028059 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:00.028063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:00.028065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:00.028322 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:00.028456 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:00.029094 (+ 638us) server_negotiation.cc:299] Negotiation successful
0808 03:56:00.029309 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 03:56:00.030575 20832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:00.012247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:41303 (local address 127.26.146.131:53209)
0808 03:56:00.012973 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:00.013013 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:00.014819 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:00.015556 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:00.015568 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:00.016337 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:00.017120 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:00.017142 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.018484 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.018488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:00.020184 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:00.020196 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.021158 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.021167 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:00.021363 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:00.021980 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:56:00.022003 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:00.024137 (+ 2134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:00.026884 (+ 2747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:00.026892 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:00.026905 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:00.027375 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:00.027707 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:00.027711 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:00.027713 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:00.027925 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:00.028443 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:00.028450 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:00.028758 (+ 308us) client_negotiation.cc:769] Sending connection context
0808 03:56:00.028976 (+ 218us) client_negotiation.cc:240] Negotiation successful
0808 03:56:00.029460 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":303,"threads_started":1}
I20260808 03:56:00.032040 20830 heartbeater.cc:347] Connected to a master server at 127.26.146.190:41303
I20260808 03:56:00.032341 20830 heartbeater.cc:464] Registering TS with master...
I20260808 03:56:00.032971 20830 heartbeater.cc:511] Master 127.26.146.190:41303 requested a full tablet report, sending...
I20260808 03:56:00.034286 20151 ts_manager.cc:194] Registered new tserver with Master: 54193318ba2f4c53b302457b4a82fdeb (127.26.146.131:37119)
I20260808 03:56:00.035045 20151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.131@KRBTEST.COM'} at 127.26.146.131:53209
I20260808 03:56:00.044174 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:56:00.060720 20438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:00.048670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55832 (local address 127.26.146.190:41303)
0808 03:56:00.048848 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:00.048854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:00.049363 (+ 509us) server_negotiation.cc:407] Connection header received
0808 03:56:00.049480 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:00.049483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:00.049556 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:00.049648 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:00.051099 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.051861 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.053129 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.053556 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.053688 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:00.055921 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:00.055926 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:00.055929 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:00.055960 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:00.058401 (+ 2441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:00.058951 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:00.058955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:00.058957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:00.059100 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:00.059436 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:00.059440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:00.059442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:00.059758 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:00.059991 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:00.060266 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 03:56:00.060423 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 03:56:00.064481 20151 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:56:00.071194 20151 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:56:00.108700 20846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:00.083192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43662 (local address 127.26.146.131:37119)
0808 03:56:00.099463 (+ 16271us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:00.099472 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:00.099510 (+ 38us) server_negotiation.cc:407] Connection header received
0808 03:56:00.099610 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:00.099619 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:00.099904 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:00.100095 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:00.101127 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.102847 (+ 1720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.106148 (+ 3301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.107544 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.107710 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:00.107921 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:00.108210 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 03:56:00.108361 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16105,"thread_start_us":15965,"threads_started":1}
I20260808 03:56:00.108707 20851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:00.082795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:37119 (local address 127.0.0.1:43662)
0808 03:56:00.097967 (+ 15172us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:00.098013 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:00.098206 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:00.100443 (+ 2237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:00.100453 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:00.100486 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:00.100906 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:00.100918 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.103564 (+ 2646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.103569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:00.105985 (+ 2416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:00.106000 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.107759 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.107767 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:00.107990 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:00.108035 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 03:56:00.108150 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 03:56:00.108360 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14901,"thread_start_us":20445,"threads_started":1}
I20260808 03:56:00.111595 20850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:00.083199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55268 (local address 127.26.146.130:36575)
0808 03:56:00.103942 (+ 20743us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:00.103950 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:00.103976 (+ 26us) server_negotiation.cc:407] Connection header received
0808 03:56:00.104069 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:00.104076 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:00.104290 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:00.104477 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:00.105331 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.106564 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.108484 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.109856 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.110005 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:00.110167 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:00.111088 (+ 921us) server_negotiation.cc:299] Negotiation successful
0808 03:56:00.111245 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20568,"thread_start_us":20451,"threads_started":1}
I20260808 03:56:00.112075 20843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:00.082933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:36575 (local address 127.0.0.1:55268)
0808 03:56:00.103024 (+ 20091us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:00.103045 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:00.103158 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:00.104719 (+ 1561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:00.104724 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:00.104752 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:00.105132 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:00.105143 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.106686 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.106691 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:00.108299 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:00.108314 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.110413 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.110421 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:00.110580 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:00.110611 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 03:56:00.110698 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 03:56:00.110821 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19958,"thread_start_us":20141,"threads_started":1}
I20260808 03:56:00.112823 20844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:00.083247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:33895 (local address 127.0.0.1:51166)
0808 03:56:00.103367 (+ 20120us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:00.103383 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:00.104373 (+ 990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:00.105320 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:00.105323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:00.105342 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:00.105722 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:00.105732 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.107499 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.107504 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:00.110269 (+ 2765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:00.110284 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.112086 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.112094 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:00.112249 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:00.112279 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 03:56:00.112380 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 03:56:00.112497 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19984,"mutex_wait_us":1,"thread_start_us":15151,"threads_started":1}
I20260808 03:56:00.113195 20852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:00.083787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51166 (local address 127.26.146.129:33895)
0808 03:56:00.094140 (+ 10353us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:00.094151 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:00.103487 (+ 9336us) server_negotiation.cc:407] Connection header received
0808 03:56:00.104613 (+ 1126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:00.104622 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:00.104843 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:00.105002 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:00.105875 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.107352 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.110462 (+ 3110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.111838 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.112030 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:00.112303 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:00.112548 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 03:56:00.112722 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10135,"thread_start_us":9937,"threads_started":1}
I20260808 03:56:00.113109 20725 tablet_service.cc:1467] Processing CreateTablet for tablet 5e79034cddad4dd39c08c9d77bd8862f (DEFAULT_TABLE table=test-table [id=71ea42cbe2344363bfd1270e370cb9da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:56:00.114818 20725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e79034cddad4dd39c08c9d77bd8862f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:00.116108 20339 tablet_service.cc:1467] Processing CreateTablet for tablet 5e79034cddad4dd39c08c9d77bd8862f (DEFAULT_TABLE table=test-table [id=71ea42cbe2344363bfd1270e370cb9da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:56:00.116204 20531 tablet_service.cc:1467] Processing CreateTablet for tablet 5e79034cddad4dd39c08c9d77bd8862f (DEFAULT_TABLE table=test-table [id=71ea42cbe2344363bfd1270e370cb9da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:56:00.117431 20339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e79034cddad4dd39c08c9d77bd8862f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:00.117560 20531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e79034cddad4dd39c08c9d77bd8862f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:00.127882 20853 tablet_bootstrap.cc:492] T 5e79034cddad4dd39c08c9d77bd8862f P b9c3fc99b06b461c91f1d24d9c29e1e3: Bootstrap starting.
I20260808 03:56:00.130851 20853 tablet_bootstrap.cc:654] T 5e79034cddad4dd39c08c9d77bd8862f P b9c3fc99b06b461c91f1d24d9c29e1e3: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:00.131994 20853 log.cc:824] T 5e79034cddad4dd39c08c9d77bd8862f P b9c3fc99b06b461c91f1d24d9c29e1e3: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:00.132088 20855 tablet_bootstrap.cc:492] T 5e79034cddad4dd39c08c9d77bd8862f P e2ddb0f966014a6e95e87d3be16ee6ef: Bootstrap starting.
I20260808 03:56:00.135159 20854 tablet_bootstrap.cc:492] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb: Bootstrap starting.
I20260808 03:56:00.136056 20855 tablet_bootstrap.cc:654] T 5e79034cddad4dd39c08c9d77bd8862f P e2ddb0f966014a6e95e87d3be16ee6ef: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:00.137288 20855 log.cc:824] T 5e79034cddad4dd39c08c9d77bd8862f P e2ddb0f966014a6e95e87d3be16ee6ef: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:00.138130 20854 tablet_bootstrap.cc:654] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:00.139313 20854 log.cc:824] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:00.140632 20855 tablet_bootstrap.cc:492] T 5e79034cddad4dd39c08c9d77bd8862f P e2ddb0f966014a6e95e87d3be16ee6ef: No bootstrap required, opened a new log
I20260808 03:56:00.140870 20855 ts_tablet_manager.cc:1397] T 5e79034cddad4dd39c08c9d77bd8862f P e2ddb0f966014a6e95e87d3be16ee6ef: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260808 03:56:00.141213 20853 tablet_bootstrap.cc:492] T 5e79034cddad4dd39c08c9d77bd8862f P b9c3fc99b06b461c91f1d24d9c29e1e3: No bootstrap required, opened a new log
I20260808 03:56:00.141482 20853 ts_tablet_manager.cc:1397] T 5e79034cddad4dd39c08c9d77bd8862f P b9c3fc99b06b461c91f1d24d9c29e1e3: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.007s
I20260808 03:56:00.142151 20854 tablet_bootstrap.cc:492] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb: No bootstrap required, opened a new log
I20260808 03:56:00.142393 20854 ts_tablet_manager.cc:1397] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.002s
I20260808 03:56:00.148195 20855 raft_consensus.cc:348] T 5e79034cddad4dd39c08c9d77bd8862f P e2ddb0f966014a6e95e87d3be16ee6ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54193318ba2f4c53b302457b4a82fdeb" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37119 } } peers { permanent_uuid: "e2ddb0f966014a6e95e87d3be16ee6ef" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 36575 } } peers { permanent_uuid: "b9c3fc99b06b461c91f1d24d9c29e1e3" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 33895 } }
I20260808 03:56:00.148521 20855 raft_consensus.cc:374] T 5e79034cddad4dd39c08c9d77bd8862f P e2ddb0f966014a6e95e87d3be16ee6ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:00.148619 20855 raft_consensus.cc:729] T 5e79034cddad4dd39c08c9d77bd8862f P e2ddb0f966014a6e95e87d3be16ee6ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2ddb0f966014a6e95e87d3be16ee6ef, State: Initialized, Role: FOLLOWER
I20260808 03:56:00.149142 20855 consensus_queue.cc:260] T 5e79034cddad4dd39c08c9d77bd8862f P e2ddb0f966014a6e95e87d3be16ee6ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54193318ba2f4c53b302457b4a82fdeb" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37119 } } peers { permanent_uuid: "e2ddb0f966014a6e95e87d3be16ee6ef" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 36575 } } peers { permanent_uuid: "b9c3fc99b06b461c91f1d24d9c29e1e3" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 33895 } }
I20260808 03:56:00.152124 20853 raft_consensus.cc:348] T 5e79034cddad4dd39c08c9d77bd8862f P b9c3fc99b06b461c91f1d24d9c29e1e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54193318ba2f4c53b302457b4a82fdeb" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37119 } } peers { permanent_uuid: "e2ddb0f966014a6e95e87d3be16ee6ef" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 36575 } } peers { permanent_uuid: "b9c3fc99b06b461c91f1d24d9c29e1e3" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 33895 } }
I20260808 03:56:00.152532 20853 raft_consensus.cc:374] T 5e79034cddad4dd39c08c9d77bd8862f P b9c3fc99b06b461c91f1d24d9c29e1e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:00.152634 20853 raft_consensus.cc:729] T 5e79034cddad4dd39c08c9d77bd8862f P b9c3fc99b06b461c91f1d24d9c29e1e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9c3fc99b06b461c91f1d24d9c29e1e3, State: Initialized, Role: FOLLOWER
I20260808 03:56:00.152439 20854 raft_consensus.cc:348] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54193318ba2f4c53b302457b4a82fdeb" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37119 } } peers { permanent_uuid: "e2ddb0f966014a6e95e87d3be16ee6ef" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 36575 } } peers { permanent_uuid: "b9c3fc99b06b461c91f1d24d9c29e1e3" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 33895 } }
I20260808 03:56:00.152856 20854 raft_consensus.cc:374] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:00.152958 20854 raft_consensus.cc:729] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54193318ba2f4c53b302457b4a82fdeb, State: Initialized, Role: FOLLOWER
I20260808 03:56:00.153322 20853 consensus_queue.cc:260] T 5e79034cddad4dd39c08c9d77bd8862f P b9c3fc99b06b461c91f1d24d9c29e1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54193318ba2f4c53b302457b4a82fdeb" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37119 } } peers { permanent_uuid: "e2ddb0f966014a6e95e87d3be16ee6ef" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 36575 } } peers { permanent_uuid: "b9c3fc99b06b461c91f1d24d9c29e1e3" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 33895 } }
I20260808 03:56:00.153486 20854 consensus_queue.cc:260] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54193318ba2f4c53b302457b4a82fdeb" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37119 } } peers { permanent_uuid: "e2ddb0f966014a6e95e87d3be16ee6ef" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 36575 } } peers { permanent_uuid: "b9c3fc99b06b461c91f1d24d9c29e1e3" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 33895 } }
I20260808 03:56:00.154585 20830 heartbeater.cc:503] Master 127.26.146.190:41303 was elected leader, sending a full tablet report...
I20260808 03:56:00.156057 20617 heartbeater.cc:503] Master 127.26.146.190:41303 was elected leader, sending a full tablet report...
I20260808 03:56:00.157011 20855 ts_tablet_manager.cc:1428] T 5e79034cddad4dd39c08c9d77bd8862f P e2ddb0f966014a6e95e87d3be16ee6ef: Time spent starting tablet: real 0.016s user 0.007s sys 0.008s
I20260808 03:56:00.158473 20853 ts_tablet_manager.cc:1428] T 5e79034cddad4dd39c08c9d77bd8862f P b9c3fc99b06b461c91f1d24d9c29e1e3: Time spent starting tablet: real 0.017s user 0.014s sys 0.003s
I20260808 03:56:00.159922 20854 ts_tablet_manager.cc:1428] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb: Time spent starting tablet: real 0.017s user 0.012s sys 0.002s
I20260808 03:56:00.212857 20864 raft_consensus.cc:482] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:56:00.213184 20864 raft_consensus.cc:504] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54193318ba2f4c53b302457b4a82fdeb" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37119 } } peers { permanent_uuid: "e2ddb0f966014a6e95e87d3be16ee6ef" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 36575 } } peers { permanent_uuid: "b9c3fc99b06b461c91f1d24d9c29e1e3" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 33895 } }
I20260808 03:56:00.215175 20864 leader_election.cc:288] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2ddb0f966014a6e95e87d3be16ee6ef (127.26.146.130:36575), b9c3fc99b06b461c91f1d24d9c29e1e3 (127.26.146.129:33895)
I20260808 03:56:00.222316 20852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:00.215995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:58191 (local address 127.26.146.129:33895)
0808 03:56:00.216177 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:00.216182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:00.216198 (+ 16us) server_negotiation.cc:407] Connection header received
0808 03:56:00.216509 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:00.216515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:00.216575 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:00.216660 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:00.217176 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.217868 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.220275 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.221451 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.221601 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:00.221670 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:00.221939 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 03:56:00.222064 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 03:56:00.222411 20865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:00.215515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:33895 (local address 127.26.146.131:58191)
0808 03:56:00.216137 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:00.216153 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:00.216249 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:00.216685 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:00.216690 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:00.216713 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:00.217027 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:00.217038 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.218008 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.218014 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:00.220077 (+ 2063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:00.220089 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.221643 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.221649 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:00.221779 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:00.221807 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 03:56:00.221887 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 03:56:00.222007 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"mutex_wait_us":45,"thread_start_us":692,"threads_started":1}
I20260808 03:56:00.223093 20850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:00.215855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:56391 (local address 127.26.146.130:36575)
0808 03:56:00.216045 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:00.216051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:00.216073 (+ 22us) server_negotiation.cc:407] Connection header received
0808 03:56:00.216156 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:00.216161 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:00.216237 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:00.216340 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:00.216751 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.217798 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.220208 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.220994 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.221099 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:00.221158 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:00.222730 (+ 1572us) server_negotiation.cc:299] Negotiation successful
0808 03:56:00.222862 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 03:56:00.223488 20832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:00.215392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:36575 (local address 127.26.146.131:56391)
0808 03:56:00.215873 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:00.215894 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:00.216029 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:00.216367 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:00.216372 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:00.216400 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:00.216585 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:00.216594 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.217967 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.217972 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:00.220048 (+ 2076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:00.220067 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.221761 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.221766 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:00.221950 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:00.221982 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 03:56:00.222062 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 03:56:00.222185 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":25}
I20260808 03:56:00.223330 20374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e79034cddad4dd39c08c9d77bd8862f" candidate_uuid: "54193318ba2f4c53b302457b4a82fdeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9c3fc99b06b461c91f1d24d9c29e1e3" is_pre_election: true
I20260808 03:56:00.223793 20374 raft_consensus.cc:2454] T 5e79034cddad4dd39c08c9d77bd8862f P b9c3fc99b06b461c91f1d24d9c29e1e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54193318ba2f4c53b302457b4a82fdeb in term 0.
I20260808 03:56:00.224476 20647 leader_election.cc:302] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54193318ba2f4c53b302457b4a82fdeb, b9c3fc99b06b461c91f1d24d9c29e1e3; no voters:
I20260808 03:56:00.224805 20864 raft_consensus.cc:2781] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:56:00.224671 20557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e79034cddad4dd39c08c9d77bd8862f" candidate_uuid: "54193318ba2f4c53b302457b4a82fdeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2ddb0f966014a6e95e87d3be16ee6ef" is_pre_election: true
I20260808 03:56:00.224912 20864 raft_consensus.cc:482] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:56:00.224977 20864 raft_consensus.cc:3037] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:00.225150 20557 raft_consensus.cc:2454] T 5e79034cddad4dd39c08c9d77bd8862f P e2ddb0f966014a6e95e87d3be16ee6ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54193318ba2f4c53b302457b4a82fdeb in term 0.
I20260808 03:56:00.227382 20864 raft_consensus.cc:504] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54193318ba2f4c53b302457b4a82fdeb" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37119 } } peers { permanent_uuid: "e2ddb0f966014a6e95e87d3be16ee6ef" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 36575 } } peers { permanent_uuid: "b9c3fc99b06b461c91f1d24d9c29e1e3" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 33895 } }
I20260808 03:56:00.228017 20864 leader_election.cc:288] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb [CANDIDATE]: Term 1 election: Requested vote from peers e2ddb0f966014a6e95e87d3be16ee6ef (127.26.146.130:36575), b9c3fc99b06b461c91f1d24d9c29e1e3 (127.26.146.129:33895)
I20260808 03:56:00.228387 20557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e79034cddad4dd39c08c9d77bd8862f" candidate_uuid: "54193318ba2f4c53b302457b4a82fdeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2ddb0f966014a6e95e87d3be16ee6ef"
I20260808 03:56:00.228593 20557 raft_consensus.cc:3037] T 5e79034cddad4dd39c08c9d77bd8862f P e2ddb0f966014a6e95e87d3be16ee6ef [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:00.228811 20374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e79034cddad4dd39c08c9d77bd8862f" candidate_uuid: "54193318ba2f4c53b302457b4a82fdeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9c3fc99b06b461c91f1d24d9c29e1e3"
I20260808 03:56:00.229015 20374 raft_consensus.cc:3037] T 5e79034cddad4dd39c08c9d77bd8862f P b9c3fc99b06b461c91f1d24d9c29e1e3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:00.231747 20557 raft_consensus.cc:2454] T 5e79034cddad4dd39c08c9d77bd8862f P e2ddb0f966014a6e95e87d3be16ee6ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54193318ba2f4c53b302457b4a82fdeb in term 1.
I20260808 03:56:00.232116 20374 raft_consensus.cc:2454] T 5e79034cddad4dd39c08c9d77bd8862f P b9c3fc99b06b461c91f1d24d9c29e1e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54193318ba2f4c53b302457b4a82fdeb in term 1.
I20260808 03:56:00.232213 20650 leader_election.cc:302] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54193318ba2f4c53b302457b4a82fdeb, e2ddb0f966014a6e95e87d3be16ee6ef; no voters:
I20260808 03:56:00.232501 20864 raft_consensus.cc:2781] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:00.232939 20864 raft_consensus.cc:686] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb [term 1 LEADER]: Becoming Leader. State: Replica: 54193318ba2f4c53b302457b4a82fdeb, State: Running, Role: LEADER
I20260808 03:56:00.233422 20864 consensus_queue.cc:237] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54193318ba2f4c53b302457b4a82fdeb" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37119 } } peers { permanent_uuid: "e2ddb0f966014a6e95e87d3be16ee6ef" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 36575 } } peers { permanent_uuid: "b9c3fc99b06b461c91f1d24d9c29e1e3" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 33895 } }
I20260808 03:56:00.237953 20150 catalog_manager.cc:5495] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb reported cstate change: term changed from 0 to 1, leader changed from <none> to 54193318ba2f4c53b302457b4a82fdeb (127.26.146.131). New cstate: current_term: 1 leader_uuid: "54193318ba2f4c53b302457b4a82fdeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54193318ba2f4c53b302457b4a82fdeb" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37119 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2ddb0f966014a6e95e87d3be16ee6ef" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 36575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9c3fc99b06b461c91f1d24d9c29e1e3" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 33895 } health_report { overall_health: UNKNOWN } } }
W20260808 03:56:00.241971 20437 tablet.cc:2369] T 5e79034cddad4dd39c08c9d77bd8862f P b9c3fc99b06b461c91f1d24d9c29e1e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:56:00.247869 20618 tablet.cc:2369] T 5e79034cddad4dd39c08c9d77bd8862f P e2ddb0f966014a6e95e87d3be16ee6ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:56:00.260656 20831 tablet.cc:2369] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:56:00.288834 20846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:00.283884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43672 (local address 127.26.146.131:37119)
0808 03:56:00.284080 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:00.284084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:00.284103 (+ 19us) server_negotiation.cc:407] Connection header received
0808 03:56:00.284282 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:00.284288 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:00.284420 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:00.284572 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:56:00.284880 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.285628 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.286947 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.287341 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.287428 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:00.287896 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:56:00.288405 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:56:00.288510 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:00.288568 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 03:56:00.288632 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 03:56:00.296442 20557 raft_consensus.cc:1261] T 5e79034cddad4dd39c08c9d77bd8862f P e2ddb0f966014a6e95e87d3be16ee6ef [term 1 FOLLOWER]: Refusing update from remote peer 54193318ba2f4c53b302457b4a82fdeb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:56:00.296438 20374 raft_consensus.cc:1261] T 5e79034cddad4dd39c08c9d77bd8862f P b9c3fc99b06b461c91f1d24d9c29e1e3 [term 1 FOLLOWER]: Refusing update from remote peer 54193318ba2f4c53b302457b4a82fdeb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:56:00.297308 20866 consensus_queue.cc:1035] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9c3fc99b06b461c91f1d24d9c29e1e3" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 33895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:56:00.297658 20864 consensus_queue.cc:1035] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2ddb0f966014a6e95e87d3be16ee6ef" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 36575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:56:00.308434 20873 mvcc.cc:205] Tried to move back new op lower bound from 7316116931765202944 to 7316116931520126976. Current Snapshot: MvccSnapshot[applied={T|T < 7316116931765202944}]
I20260808 03:56:00.319494 20874 mvcc.cc:205] Tried to move back new op lower bound from 7316116931765202944 to 7316116931520126976. Current Snapshot: MvccSnapshot[applied={T|T < 7316116931765202944}]
I20260808 03:56:00.319850 20151 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:56:00.320154 20151 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:56:00.323817 20151 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e134eed8ab7047a6884f66d8464e0fb5: Sending DeleteTablet for 3 replicas of tablet 5e79034cddad4dd39c08c9d77bd8862f
I20260808 03:56:00.324926 20725 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e79034cddad4dd39c08c9d77bd8862f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:56:00 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:43662
I20260808 03:56:00.325145 20531 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e79034cddad4dd39c08c9d77bd8862f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:56:00 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:55268
I20260808 03:56:00.325816 20884 tablet_replica.cc:330] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb: stopping tablet replica
I20260808 03:56:00.325747 20339 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e79034cddad4dd39c08c9d77bd8862f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:56:00 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:51166
I20260808 03:56:00.326273 20884 raft_consensus.cc:2229] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:56:00.326969 20884 raft_consensus.cc:2258] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:56:00.327095 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 20244
I20260808 03:56:00.331557 20884 ts_tablet_manager.cc:1905] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:56:00.336328 20885 tablet_replica.cc:330] T 5e79034cddad4dd39c08c9d77bd8862f P e2ddb0f966014a6e95e87d3be16ee6ef: stopping tablet replica
I20260808 03:56:00.336771 20885 raft_consensus.cc:2229] T 5e79034cddad4dd39c08c9d77bd8862f P e2ddb0f966014a6e95e87d3be16ee6ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:56:00.337286 20885 raft_consensus.cc:2258] T 5e79034cddad4dd39c08c9d77bd8862f P e2ddb0f966014a6e95e87d3be16ee6ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:56:00.341001 20884 ts_tablet_manager.cc:1918] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:56:00.341169 20884 log.cc:1196] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/5e79034cddad4dd39c08c9d77bd8862f
I20260808 03:56:00.341735 20884 ts_tablet_manager.cc:1939] T 5e79034cddad4dd39c08c9d77bd8862f P 54193318ba2f4c53b302457b4a82fdeb: Deleting consensus metadata
I20260808 03:56:00.342104 20888 tablet_replica.cc:330] T 5e79034cddad4dd39c08c9d77bd8862f P b9c3fc99b06b461c91f1d24d9c29e1e3: stopping tablet replica
I20260808 03:56:00.342623 20888 raft_consensus.cc:2229] T 5e79034cddad4dd39c08c9d77bd8862f P b9c3fc99b06b461c91f1d24d9c29e1e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:56:00.343243 20888 raft_consensus.cc:2258] T 5e79034cddad4dd39c08c9d77bd8862f P b9c3fc99b06b461c91f1d24d9c29e1e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:56:00.343485 20131 catalog_manager.cc:4841] TS 54193318ba2f4c53b302457b4a82fdeb (127.26.146.131:37119): tablet 5e79034cddad4dd39c08c9d77bd8862f (table test-table [id=71ea42cbe2344363bfd1270e370cb9da]) successfully deleted
I20260808 03:56:00.345592 20888 ts_tablet_manager.cc:1905] T 5e79034cddad4dd39c08c9d77bd8862f P b9c3fc99b06b461c91f1d24d9c29e1e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:56:00.346078 20885 ts_tablet_manager.cc:1905] T 5e79034cddad4dd39c08c9d77bd8862f P e2ddb0f966014a6e95e87d3be16ee6ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:56:00.352596 20888 ts_tablet_manager.cc:1918] T 5e79034cddad4dd39c08c9d77bd8862f P b9c3fc99b06b461c91f1d24d9c29e1e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:56:00.352800 20888 log.cc:1196] T 5e79034cddad4dd39c08c9d77bd8862f P b9c3fc99b06b461c91f1d24d9c29e1e3: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-0/wal/wals/5e79034cddad4dd39c08c9d77bd8862f
I20260808 03:56:00.353128 20885 ts_tablet_manager.cc:1918] T 5e79034cddad4dd39c08c9d77bd8862f P e2ddb0f966014a6e95e87d3be16ee6ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:56:00.353283 20885 log.cc:1196] T 5e79034cddad4dd39c08c9d77bd8862f P e2ddb0f966014a6e95e87d3be16ee6ef: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/5e79034cddad4dd39c08c9d77bd8862f
I20260808 03:56:00.353551 20888 ts_tablet_manager.cc:1939] T 5e79034cddad4dd39c08c9d77bd8862f P b9c3fc99b06b461c91f1d24d9c29e1e3: Deleting consensus metadata
I20260808 03:56:00.353826 20885 ts_tablet_manager.cc:1939] T 5e79034cddad4dd39c08c9d77bd8862f P e2ddb0f966014a6e95e87d3be16ee6ef: Deleting consensus metadata
I20260808 03:56:00.355355 20130 catalog_manager.cc:4841] TS b9c3fc99b06b461c91f1d24d9c29e1e3 (127.26.146.129:33895): tablet 5e79034cddad4dd39c08c9d77bd8862f (table test-table [id=71ea42cbe2344363bfd1270e370cb9da]) successfully deleted
Aug 08 03:56:00 dist-test-slave-jlzn krb5kdc[20086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.129@KRBTEST.COM
I20260808 03:56:00.355599 20133 catalog_manager.cc:4841] TS e2ddb0f966014a6e95e87d3be16ee6ef (127.26.146.130:36575): tablet 5e79034cddad4dd39c08c9d77bd8862f (table test-table [id=71ea42cbe2344363bfd1270e370cb9da]) successfully deleted
I20260808 03:56:00.362089 20852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:00.328287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51170 (local address 127.26.146.129:33895)
0808 03:56:00.328452 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:00.328457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:00.344420 (+ 15963us) server_negotiation.cc:407] Connection header received
0808 03:56:00.344623 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:00.344628 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:00.344712 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:00.344826 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:00.346706 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.347974 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.350082 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.350806 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.350917 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:00.356689 (+ 5772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:00.356699 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:00.356713 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:00.356743 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:00.359349 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:00.360248 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:00.360254 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:00.360257 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:00.360388 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:00.360793 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:00.360795 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:00.360797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:00.361113 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:00.361337 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:00.361667 (+ 330us) server_negotiation.cc:299] Negotiation successful
0808 03:56:00.361829 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 03:56:00.362800 20428 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:00.457362 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 20244
I20260808 03:56:00.489676 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 20445
Aug 08 03:56:00 dist-test-slave-jlzn krb5kdc[20086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.130@KRBTEST.COM
I20260808 03:56:00.508863 20850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:00.490566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55284 (local address 127.26.146.130:36575)
0808 03:56:00.490734 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:00.490739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:00.490757 (+ 18us) server_negotiation.cc:407] Connection header received
0808 03:56:00.490910 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:00.490913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:00.490975 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:00.491053 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:00.492766 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.493498 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.495359 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.496014 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.496165 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:00.501563 (+ 5398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:00.501578 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:00.501596 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:00.501646 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:00.505607 (+ 3961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:00.506479 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:00.506490 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:00.506494 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:00.506654 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:00.507097 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:00.507102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:00.507111 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:00.507639 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:00.507979 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:00.508259 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 03:56:00.508492 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 03:56:00.509586 20610 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:00.606218 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 20445
I20260808 03:56:00.629465 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 20624
Aug 08 03:56:00 dist-test-slave-jlzn krb5kdc[20086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.131@KRBTEST.COM
I20260808 03:56:00.645134 20846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:00.630228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43686 (local address 127.26.146.131:37119)
0808 03:56:00.630424 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:00.630430 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:00.630454 (+ 24us) server_negotiation.cc:407] Connection header received
0808 03:56:00.630661 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:00.630666 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:00.630730 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:00.630814 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:00.632121 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.632956 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.634239 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:00.634695 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:00.634813 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:00.639228 (+ 4415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:00.639240 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:00.639257 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:00.639304 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:00.642274 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:00.642905 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:00.642913 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:00.642915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:00.643081 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:00.643448 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:00.643453 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:00.643455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:00.643993 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:00.644327 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:00.644562 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 03:56:00.644786 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 03:56:00.645798 20822 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:00.767027 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 20624
I20260808 03:56:00.792477 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 20102
I20260808 03:56:00.793740 20194 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:00.904239 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 20102
2026-08-08T03:56:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4471 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T03:56:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:56:00Z Disabled control of system clock
I20260808 03:56:00.996616 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:41557
--webserver_interface=127.26.146.190
--webserver_port=0
--builtin_ntp_servers=127.26.146.148:33945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:41557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 03:56:01.323324 20900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:01.323614 20900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:01.323720 20900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:01.331080 20900 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:01.331189 20900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:01.331234 20900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:01.331321 20900 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:01.331377 20900 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:01.340425 20900 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:33945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:41557
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:41557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:01.341691 20900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:01.343446 20900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:01.354645 20906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:01.354601 20909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:01.354678 20907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:01.355449 20900 server_base.cc:1034] running on GCE node
I20260808 03:56:01.356232 20900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:01.357518 20900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:01.358707 20900 hybrid_clock.cc:648] HybridClock initialized: now 1786161361358666 us; error 30 us; skew 500 ppm
I20260808 03:56:01.359030 20900 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:56:01.365689 20900 webserver.cc:460] Webserver started at http://127.26.146.190:41111/ using document root <none> and password file <none>
I20260808 03:56:01.366394 20900 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:01.366501 20900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:01.366843 20900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:01.369777 20900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "47087fa26a3f4c7593f8a4ec7b73ac7e"
format_stamp: "Formatted at 2026-08-08 03:56:01 on dist-test-slave-jlzn"
server_key: "7ecb15441cc11675a71b8134f58957ae"
server_key_iv: "ddf86b1d746ab15db4d7732eb0e876e8"
server_key_version: "encryptionkey@0"
I20260808 03:56:01.370559 20900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "47087fa26a3f4c7593f8a4ec7b73ac7e"
format_stamp: "Formatted at 2026-08-08 03:56:01 on dist-test-slave-jlzn"
server_key: "7ecb15441cc11675a71b8134f58957ae"
server_key_iv: "ddf86b1d746ab15db4d7732eb0e876e8"
server_key_version: "encryptionkey@0"
I20260808 03:56:01.375519 20900 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 03:56:01.378901 20915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:01.380213 20900 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:56:01.380371 20900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "47087fa26a3f4c7593f8a4ec7b73ac7e"
format_stamp: "Formatted at 2026-08-08 03:56:01 on dist-test-slave-jlzn"
server_key: "7ecb15441cc11675a71b8134f58957ae"
server_key_iv: "ddf86b1d746ab15db4d7732eb0e876e8"
server_key_version: "encryptionkey@0"
I20260808 03:56:01.380560 20900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:01.395247 20900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:01.396024 20900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:01.396291 20900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:01.413360 20900 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:41557
I20260808 03:56:01.413393 20992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:41557 every 8 connection(s)
I20260808 03:56:01.414654 20900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:56:01.418319 20993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:01.421039 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 20900
I20260808 03:56:01.421219 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:56:01.421797 27210 external_mini_cluster.cc:1428] Setting key 54e13f6e36eb3c5f8d31ab1edfa37d84
I20260808 03:56:01.429863 20993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47087fa26a3f4c7593f8a4ec7b73ac7e: Bootstrap starting.
I20260808 03:56:01.432850 20993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47087fa26a3f4c7593f8a4ec7b73ac7e: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:01.433943 20993 log.cc:824] T 00000000000000000000000000000000 P 47087fa26a3f4c7593f8a4ec7b73ac7e: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:01.434934 20996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:01.423722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48366 (local address 127.26.146.190:41557)
0808 03:56:01.424728 (+ 1006us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:01.424738 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:01.424778 (+ 40us) server_negotiation.cc:407] Connection header received
0808 03:56:01.426376 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:01.426397 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:01.426825 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:01.427177 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:01.427755 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:01.429051 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:01.430535 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:01.431199 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:01.431366 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:01.432500 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:01.432507 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:01.432521 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:01.433071 (+ 550us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:01.433168 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:01.433192 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:01.433304 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:01.433519 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 03:56:01.433806 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":374,"threads_started":1}
I20260808 03:56:01.436962 20993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47087fa26a3f4c7593f8a4ec7b73ac7e: No bootstrap required, opened a new log
I20260808 03:56:01.444531 20993 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47087fa26a3f4c7593f8a4ec7b73ac7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47087fa26a3f4c7593f8a4ec7b73ac7e" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 41557 } }
I20260808 03:56:01.444859 20993 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47087fa26a3f4c7593f8a4ec7b73ac7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:01.444932 20993 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47087fa26a3f4c7593f8a4ec7b73ac7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47087fa26a3f4c7593f8a4ec7b73ac7e, State: Initialized, Role: FOLLOWER
I20260808 03:56:01.445408 20993 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47087fa26a3f4c7593f8a4ec7b73ac7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47087fa26a3f4c7593f8a4ec7b73ac7e" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 41557 } }
I20260808 03:56:01.445607 20993 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47087fa26a3f4c7593f8a4ec7b73ac7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:56:01.445703 20993 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47087fa26a3f4c7593f8a4ec7b73ac7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:56:01.445818 20993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47087fa26a3f4c7593f8a4ec7b73ac7e [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:01.447839 20993 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47087fa26a3f4c7593f8a4ec7b73ac7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47087fa26a3f4c7593f8a4ec7b73ac7e" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 41557 } }
I20260808 03:56:01.448284 20993 leader_election.cc:302] T 00000000000000000000000000000000 P 47087fa26a3f4c7593f8a4ec7b73ac7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47087fa26a3f4c7593f8a4ec7b73ac7e; no voters:
I20260808 03:56:01.448782 20993 leader_election.cc:288] T 00000000000000000000000000000000 P 47087fa26a3f4c7593f8a4ec7b73ac7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:56:01.448920 20998 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47087fa26a3f4c7593f8a4ec7b73ac7e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:01.449683 20998 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47087fa26a3f4c7593f8a4ec7b73ac7e [term 1 LEADER]: Becoming Leader. State: Replica: 47087fa26a3f4c7593f8a4ec7b73ac7e, State: Running, Role: LEADER
I20260808 03:56:01.450170 20998 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47087fa26a3f4c7593f8a4ec7b73ac7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47087fa26a3f4c7593f8a4ec7b73ac7e" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 41557 } }
I20260808 03:56:01.450838 20993 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47087fa26a3f4c7593f8a4ec7b73ac7e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:01.452975 20999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47087fa26a3f4c7593f8a4ec7b73ac7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47087fa26a3f4c7593f8a4ec7b73ac7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47087fa26a3f4c7593f8a4ec7b73ac7e" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 41557 } } }
I20260808 03:56:01.453387 20999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47087fa26a3f4c7593f8a4ec7b73ac7e [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:01.453715 21000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47087fa26a3f4c7593f8a4ec7b73ac7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47087fa26a3f4c7593f8a4ec7b73ac7e. Latest consensus state: current_term: 1 leader_uuid: "47087fa26a3f4c7593f8a4ec7b73ac7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47087fa26a3f4c7593f8a4ec7b73ac7e" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 41557 } } }
I20260808 03:56:01.454043 21000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47087fa26a3f4c7593f8a4ec7b73ac7e [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:01.455391 21013 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:56:01.460390 21013 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:56:01.469645 21013 catalog_manager.cc:1347] Generated new cluster ID: 7d04b8cf82384eed8d947724f221566a
I20260808 03:56:01.469751 21013 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:56:01.489161 21013 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:56:01.491025 21013 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:56:01.500960 21013 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47087fa26a3f4c7593f8a4ec7b73ac7e: Generated new TSK 0
I20260808 03:56:01.501773 21013 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:56:01.515437 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:41557
--builtin_ntp_servers=127.26.146.148:33945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 03:56:01.865319 21027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:01.865592 21027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:01.865720 21027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:01.873111 21027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:01.873216 21027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:01.873358 21027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:56:01.882691 21027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:33945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:41557
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:01.884102 21027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:01.885934 21027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:01.898721 21036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:01.898763 21038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:01.898689 21040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:01.899745 21027 server_base.cc:1034] running on GCE node
I20260808 03:56:01.900357 21027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:01.901239 21027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:01.902462 21027 hybrid_clock.cc:648] HybridClock initialized: now 1786161361902410 us; error 38 us; skew 500 ppm
I20260808 03:56:01.902773 21027 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:56:01.904995 21027 webserver.cc:460] Webserver started at http://127.26.146.129:46005/ using document root <none> and password file <none>
I20260808 03:56:01.905653 21027 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:01.905758 21027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:01.906057 21027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:01.909027 21027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "a8eb5938cc474f7fbc798f0dafe6d902"
format_stamp: "Formatted at 2026-08-08 03:56:01 on dist-test-slave-jlzn"
server_key: "5fcc3f8bdf332c687006b0f4b0b617fe"
server_key_iv: "371a1303f93105767c54bfb90df7485a"
server_key_version: "encryptionkey@0"
I20260808 03:56:01.909811 21027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "a8eb5938cc474f7fbc798f0dafe6d902"
format_stamp: "Formatted at 2026-08-08 03:56:01 on dist-test-slave-jlzn"
server_key: "5fcc3f8bdf332c687006b0f4b0b617fe"
server_key_iv: "371a1303f93105767c54bfb90df7485a"
server_key_version: "encryptionkey@0"
I20260808 03:56:01.914819 21027 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 03:56:01.918155 21050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:01.919461 21027 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 03:56:01.919667 21027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "a8eb5938cc474f7fbc798f0dafe6d902"
format_stamp: "Formatted at 2026-08-08 03:56:01 on dist-test-slave-jlzn"
server_key: "5fcc3f8bdf332c687006b0f4b0b617fe"
server_key_iv: "371a1303f93105767c54bfb90df7485a"
server_key_version: "encryptionkey@0"
I20260808 03:56:01.919894 21027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:01.958618 21027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:01.959432 21027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:01.959730 21027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:01.960703 21027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:56:01.962162 21027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:56:01.962252 21027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:01.962333 21027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:56:01.962385 21027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:01.992113 21027 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:37177
I20260808 03:56:01.992166 21205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:37177 every 8 connection(s)
I20260808 03:56:01.993350 21027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:56:02.000285 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 21027
I20260808 03:56:02.000456 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:56:02.004573 27210 external_mini_cluster.cc:1428] Setting key 75e615a1f51906425a2c9ade9a9c3dd4
I20260808 03:56:02.005563 21209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:01.995764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:40501 (local address 127.26.146.190:41557)
0808 03:56:01.996214 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:01.996218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:01.997366 (+ 1148us) server_negotiation.cc:407] Connection header received
0808 03:56:01.999112 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:01.999117 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:01.999194 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:01.999287 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:02.000446 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.001292 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.002818 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.003253 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.003363 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:02.004700 (+ 1337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:02.004706 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:02.004711 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:02.004733 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:02.004779 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:02.004792 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:02.004867 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:02.005082 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 03:56:02.005190 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":210,"threads_started":1}
I20260808 03:56:02.006512 21210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:01.996335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:41557 (local address 127.26.146.129:40501)
0808 03:56:01.997211 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:01.997257 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:01.998854 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:01.999515 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:01.999524 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:01.999611 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:02.000260 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:02.000287 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.001446 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.001449 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:02.002678 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:02.002691 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.003469 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.003476 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:02.003691 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:02.004392 (+ 701us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:56:02.004395 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:02.004434 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:56:02.004511 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:02.004881 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:02.004893 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:02.004900 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 03:56:02.005040 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 03:56:02.005423 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":251,"threads_started":1}
I20260808 03:56:02.007258 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:41557
--builtin_ntp_servers=127.26.146.148:33945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 03:56:02.007934 21207 heartbeater.cc:347] Connected to a master server at 127.26.146.190:41557
I20260808 03:56:02.008253 21207 heartbeater.cc:464] Registering TS with master...
I20260808 03:56:02.009110 21207 heartbeater.cc:511] Master 127.26.146.190:41557 requested a full tablet report, sending...
I20260808 03:56:02.011330 20941 ts_manager.cc:194] Registered new tserver with Master: a8eb5938cc474f7fbc798f0dafe6d902 (127.26.146.129:37177)
I20260808 03:56:02.012852 20941 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.146.129:40501
W20260808 03:56:02.332072 21211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:02.332334 21211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:02.332420 21211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:02.339354 21211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:02.339449 21211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:02.339555 21211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:56:02.348182 21211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:33945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:41557
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:02.349363 21211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:02.351190 21211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:02.363480 21216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:02.363595 21217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:02.363526 21219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:02.364355 21211 server_base.cc:1034] running on GCE node
I20260808 03:56:02.364902 21211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:02.365773 21211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:02.366977 21211 hybrid_clock.cc:648] HybridClock initialized: now 1786161362366921 us; error 46 us; skew 500 ppm
I20260808 03:56:02.367268 21211 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:56:02.369401 21211 webserver.cc:460] Webserver started at http://127.26.146.130:32825/ using document root <none> and password file <none>
I20260808 03:56:02.370025 21211 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:02.370126 21211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:02.370422 21211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:02.373637 21211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "385e80f2a3af46b58e98fe92c340dc96"
format_stamp: "Formatted at 2026-08-08 03:56:02 on dist-test-slave-jlzn"
server_key: "560b00963809c3af6def17f51b180d46"
server_key_iv: "64766450753c538b551be8240fcb4c39"
server_key_version: "encryptionkey@0"
I20260808 03:56:02.374449 21211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "385e80f2a3af46b58e98fe92c340dc96"
format_stamp: "Formatted at 2026-08-08 03:56:02 on dist-test-slave-jlzn"
server_key: "560b00963809c3af6def17f51b180d46"
server_key_iv: "64766450753c538b551be8240fcb4c39"
server_key_version: "encryptionkey@0"
I20260808 03:56:02.379292 21211 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 03:56:02.382522 21227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:02.383690 21211 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 03:56:02.383905 21211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "385e80f2a3af46b58e98fe92c340dc96"
format_stamp: "Formatted at 2026-08-08 03:56:02 on dist-test-slave-jlzn"
server_key: "560b00963809c3af6def17f51b180d46"
server_key_iv: "64766450753c538b551be8240fcb4c39"
server_key_version: "encryptionkey@0"
I20260808 03:56:02.384119 21211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:02.396178 21211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:02.396884 21211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:02.397167 21211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:02.398026 21211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:56:02.399401 21211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:56:02.399502 21211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:02.399611 21211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:56:02.399688 21211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:02.437520 21211 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:38123
I20260808 03:56:02.437565 21390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:38123 every 8 connection(s)
I20260808 03:56:02.438776 21211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:56:02.445704 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 21211
I20260808 03:56:02.445885 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:56:02.446473 27210 external_mini_cluster.cc:1428] Setting key 7c212abc1223e98547c53ddf3132276c
I20260808 03:56:02.449483 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:41557
--builtin_ntp_servers=127.26.146.148:33945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 03:56:02.451148 21209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:02.441194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:47867 (local address 127.26.146.190:41557)
0808 03:56:02.441376 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:02.441380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:02.442659 (+ 1279us) server_negotiation.cc:407] Connection header received
0808 03:56:02.444340 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:02.444344 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:02.444410 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:02.444493 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:02.445448 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.446505 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.448495 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.449094 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.449227 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:02.450418 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:02.450425 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:02.450429 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:02.450450 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:02.450489 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:02.450504 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:02.450590 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:02.450767 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 03:56:02.450859 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 03:56:02.452155 21394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:02.441678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:41557 (local address 127.26.146.130:47867)
0808 03:56:02.442512 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:02.442546 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:02.444119 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:02.444670 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:02.444677 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:02.444758 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:02.445289 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:02.445304 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.446670 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.446674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:02.448355 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:02.448370 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.449277 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.449282 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:02.449462 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:02.450128 (+ 666us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:56:02.450131 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:02.450168 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:56:02.450237 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:02.450594 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:02.450602 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:02.450607 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 03:56:02.450738 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 03:56:02.451070 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":266,"threads_started":1}
I20260808 03:56:02.453418 21391 heartbeater.cc:347] Connected to a master server at 127.26.146.190:41557
I20260808 03:56:02.453698 21391 heartbeater.cc:464] Registering TS with master...
I20260808 03:56:02.454382 21391 heartbeater.cc:511] Master 127.26.146.190:41557 requested a full tablet report, sending...
I20260808 03:56:02.455747 20941 ts_manager.cc:194] Registered new tserver with Master: 385e80f2a3af46b58e98fe92c340dc96 (127.26.146.130:38123)
I20260808 03:56:02.456591 20941 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.146.130:47867
W20260808 03:56:02.779559 21397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:02.779891 21397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:02.780032 21397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:02.787384 21397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:02.787489 21397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:02.787639 21397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:56:02.796592 21397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:33945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:41557
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:02.797835 21397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:02.799702 21397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:02.812672 21402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:02.812706 21405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:02.812765 21403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:02.813506 21397 server_base.cc:1034] running on GCE node
I20260808 03:56:02.814064 21397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:02.814946 21397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:02.816154 21397 hybrid_clock.cc:648] HybridClock initialized: now 1786161362816102 us; error 42 us; skew 500 ppm
I20260808 03:56:02.816460 21397 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:56:02.818632 21397 webserver.cc:460] Webserver started at http://127.26.146.131:38429/ using document root <none> and password file <none>
I20260808 03:56:02.819294 21397 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:02.819394 21397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:02.819702 21397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:02.822563 21397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "b4009e96862048b79ecfd03ca827523d"
format_stamp: "Formatted at 2026-08-08 03:56:02 on dist-test-slave-jlzn"
server_key: "e6a9daf6c1a256b4eebd1c747cee8062"
server_key_iv: "8141ef8688f60ec5aaa6ef9d186f5cc6"
server_key_version: "encryptionkey@0"
I20260808 03:56:02.823349 21397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "b4009e96862048b79ecfd03ca827523d"
format_stamp: "Formatted at 2026-08-08 03:56:02 on dist-test-slave-jlzn"
server_key: "e6a9daf6c1a256b4eebd1c747cee8062"
server_key_iv: "8141ef8688f60ec5aaa6ef9d186f5cc6"
server_key_version: "encryptionkey@0"
I20260808 03:56:02.828282 21397 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 03:56:02.831540 21412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:02.832846 21397 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:56:02.833011 21397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "b4009e96862048b79ecfd03ca827523d"
format_stamp: "Formatted at 2026-08-08 03:56:02 on dist-test-slave-jlzn"
server_key: "e6a9daf6c1a256b4eebd1c747cee8062"
server_key_iv: "8141ef8688f60ec5aaa6ef9d186f5cc6"
server_key_version: "encryptionkey@0"
I20260808 03:56:02.833214 21397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:02.849648 21397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:02.850430 21397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:02.850728 21397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:02.851702 21397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:56:02.853505 21397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:56:02.853617 21397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:02.853722 21397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:56:02.853796 21397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:02.885674 21397 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:35237
I20260808 03:56:02.885725 21579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:35237 every 8 connection(s)
I20260808 03:56:02.887030 21397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:56:02.887565 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 21397
I20260808 03:56:02.887766 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:56:02.888384 27210 external_mini_cluster.cc:1428] Setting key cc83f0dceb887c9ec497365e56c4aa48
I20260808 03:56:02.899405 21209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:02.889926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:48227 (local address 127.26.146.190:41557)
0808 03:56:02.890123 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:02.890140 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:02.891688 (+ 1548us) server_negotiation.cc:407] Connection header received
0808 03:56:02.893412 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:02.893416 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:02.893483 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:02.893577 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:02.894686 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.895440 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.896948 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.897353 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.897445 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:02.898660 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:02.898665 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:02.898669 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:02.898686 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:02.898739 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:02.898753 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:02.898840 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:02.899056 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 03:56:02.899143 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 03:56:02.900275 21582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:02.890516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:41557 (local address 127.26.146.131:48227)
0808 03:56:02.891523 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:02.891579 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:02.893177 (+ 1598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:02.893757 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:02.893764 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:02.893845 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:02.894518 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:02.894543 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.895565 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.895568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:02.896812 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:02.896824 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.897521 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.897525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:02.897713 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:02.898328 (+ 615us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:56:02.898331 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:02.898368 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:56:02.898439 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:02.898842 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:02.898849 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:02.898853 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 03:56:02.898981 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 03:56:02.899267 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":302,"threads_started":1}
I20260808 03:56:02.901597 21580 heartbeater.cc:347] Connected to a master server at 127.26.146.190:41557
I20260808 03:56:02.901887 21580 heartbeater.cc:464] Registering TS with master...
I20260808 03:56:02.902659 21580 heartbeater.cc:511] Master 127.26.146.190:41557 requested a full tablet report, sending...
I20260808 03:56:02.904178 20941 ts_manager.cc:194] Registered new tserver with Master: b4009e96862048b79ecfd03ca827523d (127.26.146.131:35237)
I20260808 03:56:02.905005 20941 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.146.131:48227
I20260808 03:56:02.915391 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:56:02.924804 21209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:02.919241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58890 (local address 127.26.146.190:41557)
0808 03:56:02.919427 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:02.919433 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:02.919787 (+ 354us) server_negotiation.cc:407] Connection header received
0808 03:56:02.919970 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:02.919974 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:02.920061 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:02.920135 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:02.920581 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.921273 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.922586 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.922967 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.923067 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:02.924093 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:02.924098 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:02.924102 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:02.924118 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:02.924164 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:02.924177 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:02.924252 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:02.924364 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 03:56:02.924453 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 03:56:02.928539 20941 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58890:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:56:02.935240 20941 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:56:02.968176 21598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:02.947450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:38123 (local address 127.0.0.1:42724)
0808 03:56:02.959413 (+ 11963us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:02.959430 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:02.959547 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:02.960562 (+ 1015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:02.960570 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:02.960598 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:02.960995 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:02.961009 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.962878 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.962883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:02.965388 (+ 2505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:02.965407 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.967195 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.967201 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:02.967394 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:02.967491 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 03:56:02.967623 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 03:56:02.967826 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11830,"thread_start_us":10699,"threads_started":1}
I20260808 03:56:02.968187 21603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:02.947890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42724 (local address 127.26.146.130:38123)
0808 03:56:02.958024 (+ 10134us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:02.958034 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:02.959672 (+ 1638us) server_negotiation.cc:407] Connection header received
0808 03:56:02.959787 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:02.959794 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:02.960101 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:02.960301 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:02.961203 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.962707 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.965600 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.967039 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.967180 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:02.967408 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:02.967673 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 03:56:02.967856 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9982,"thread_start_us":9867,"threads_started":1}
I20260808 03:56:02.968334 21599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:02.947276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:37177 (local address 127.0.0.1:41310)
0808 03:56:02.958575 (+ 11299us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:02.958621 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:02.958823 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:02.961215 (+ 2392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:02.961219 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:02.961245 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:02.961659 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:02.961668 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.963721 (+ 2053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.963726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:02.965803 (+ 2077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:02.965813 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.967490 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.967496 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:02.967690 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:02.967724 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 03:56:02.967787 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 03:56:02.967948 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11006,"mutex_wait_us":19,"thread_start_us":11840,"threads_started":1}
I20260808 03:56:02.969491 21601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:02.947482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41310 (local address 127.26.146.129:37177)
0808 03:56:02.960655 (+ 13173us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:02.960666 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:02.960704 (+ 38us) server_negotiation.cc:407] Connection header received
0808 03:56:02.960797 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:02.960804 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:02.961019 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:02.961443 (+ 424us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:02.961811 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.963589 (+ 1778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.966018 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.967277 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.968665 (+ 1388us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:02.968863 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:02.968975 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 03:56:02.969148 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12979,"thread_start_us":12857,"threads_started":1}
I20260808 03:56:02.969584 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:02.949550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:35237 (local address 127.0.0.1:59340)
0808 03:56:02.961929 (+ 12379us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:02.961942 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:02.962042 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:02.962829 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:02.962833 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:02.962858 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:02.963314 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:02.963326 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.964698 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.964703 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:02.966882 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:02.966896 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.968890 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.968896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:02.969055 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:02.969084 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 03:56:02.969157 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 03:56:02.969274 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12298,"mutex_wait_us":21,"thread_start_us":11738,"threads_started":1}
I20260808 03:56:02.969919 21605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:02.950116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59340 (local address 127.26.146.131:35237)
0808 03:56:02.961130 (+ 11014us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:02.961138 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:02.962045 (+ 907us) server_negotiation.cc:407] Connection header received
0808 03:56:02.962174 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:02.962179 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:02.962377 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:02.962552 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:02.963493 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.964561 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.967088 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:02.968703 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:02.969009 (+ 306us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:02.969220 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:02.969288 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 03:56:02.969424 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10844,"thread_start_us":10686,"threads_started":1}
I20260808 03:56:02.972731 21110 tablet_service.cc:1467] Processing CreateTablet for tablet 19a4c91e9f3347f2a302b3f1cbe2813f (DEFAULT_TABLE table=test-table [id=868e37a98ad24d968f90a124e5f77936]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:56:02.972519 21294 tablet_service.cc:1467] Processing CreateTablet for tablet 19a4c91e9f3347f2a302b3f1cbe2813f (DEFAULT_TABLE table=test-table [id=868e37a98ad24d968f90a124e5f77936]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:56:02.973866 21110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19a4c91e9f3347f2a302b3f1cbe2813f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:02.973942 21294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19a4c91e9f3347f2a302b3f1cbe2813f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:02.973897 21480 tablet_service.cc:1467] Processing CreateTablet for tablet 19a4c91e9f3347f2a302b3f1cbe2813f (DEFAULT_TABLE table=test-table [id=868e37a98ad24d968f90a124e5f77936]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:56:02.975231 21480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19a4c91e9f3347f2a302b3f1cbe2813f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:02.984164 21606 tablet_bootstrap.cc:492] T 19a4c91e9f3347f2a302b3f1cbe2813f P a8eb5938cc474f7fbc798f0dafe6d902: Bootstrap starting.
I20260808 03:56:02.987769 21609 tablet_bootstrap.cc:492] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96: Bootstrap starting.
I20260808 03:56:02.988227 21606 tablet_bootstrap.cc:654] T 19a4c91e9f3347f2a302b3f1cbe2813f P a8eb5938cc474f7fbc798f0dafe6d902: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:02.989240 21610 tablet_bootstrap.cc:492] T 19a4c91e9f3347f2a302b3f1cbe2813f P b4009e96862048b79ecfd03ca827523d: Bootstrap starting.
I20260808 03:56:02.990039 21606 log.cc:824] T 19a4c91e9f3347f2a302b3f1cbe2813f P a8eb5938cc474f7fbc798f0dafe6d902: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:02.991295 21609 tablet_bootstrap.cc:654] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:02.992527 21610 tablet_bootstrap.cc:654] T 19a4c91e9f3347f2a302b3f1cbe2813f P b4009e96862048b79ecfd03ca827523d: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:02.992602 21609 log.cc:824] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:02.993642 21610 log.cc:824] T 19a4c91e9f3347f2a302b3f1cbe2813f P b4009e96862048b79ecfd03ca827523d: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:02.999766 21606 tablet_bootstrap.cc:492] T 19a4c91e9f3347f2a302b3f1cbe2813f P a8eb5938cc474f7fbc798f0dafe6d902: No bootstrap required, opened a new log
I20260808 03:56:03.000185 21606 ts_tablet_manager.cc:1397] T 19a4c91e9f3347f2a302b3f1cbe2813f P a8eb5938cc474f7fbc798f0dafe6d902: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.006s
I20260808 03:56:03.007442 21610 tablet_bootstrap.cc:492] T 19a4c91e9f3347f2a302b3f1cbe2813f P b4009e96862048b79ecfd03ca827523d: No bootstrap required, opened a new log
I20260808 03:56:03.007750 21610 ts_tablet_manager.cc:1397] T 19a4c91e9f3347f2a302b3f1cbe2813f P b4009e96862048b79ecfd03ca827523d: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.008s
I20260808 03:56:03.008415 21609 tablet_bootstrap.cc:492] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96: No bootstrap required, opened a new log
I20260808 03:56:03.008786 21609 ts_tablet_manager.cc:1397] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96: Time spent bootstrapping tablet: real 0.021s user 0.012s sys 0.007s
I20260808 03:56:03.011363 21606 raft_consensus.cc:348] T 19a4c91e9f3347f2a302b3f1cbe2813f P a8eb5938cc474f7fbc798f0dafe6d902 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eb5938cc474f7fbc798f0dafe6d902" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37177 } } peers { permanent_uuid: "385e80f2a3af46b58e98fe92c340dc96" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38123 } } peers { permanent_uuid: "b4009e96862048b79ecfd03ca827523d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 35237 } }
I20260808 03:56:03.011741 21606 raft_consensus.cc:374] T 19a4c91e9f3347f2a302b3f1cbe2813f P a8eb5938cc474f7fbc798f0dafe6d902 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:03.011879 21606 raft_consensus.cc:729] T 19a4c91e9f3347f2a302b3f1cbe2813f P a8eb5938cc474f7fbc798f0dafe6d902 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8eb5938cc474f7fbc798f0dafe6d902, State: Initialized, Role: FOLLOWER
I20260808 03:56:03.012465 21606 consensus_queue.cc:260] T 19a4c91e9f3347f2a302b3f1cbe2813f P a8eb5938cc474f7fbc798f0dafe6d902 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eb5938cc474f7fbc798f0dafe6d902" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37177 } } peers { permanent_uuid: "385e80f2a3af46b58e98fe92c340dc96" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38123 } } peers { permanent_uuid: "b4009e96862048b79ecfd03ca827523d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 35237 } }
I20260808 03:56:03.013506 21207 heartbeater.cc:503] Master 127.26.146.190:41557 was elected leader, sending a full tablet report...
I20260808 03:56:03.014024 21606 ts_tablet_manager.cc:1428] T 19a4c91e9f3347f2a302b3f1cbe2813f P a8eb5938cc474f7fbc798f0dafe6d902: Time spent starting tablet: real 0.014s user 0.005s sys 0.007s
I20260808 03:56:03.018170 21609 raft_consensus.cc:348] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eb5938cc474f7fbc798f0dafe6d902" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37177 } } peers { permanent_uuid: "385e80f2a3af46b58e98fe92c340dc96" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38123 } } peers { permanent_uuid: "b4009e96862048b79ecfd03ca827523d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 35237 } }
I20260808 03:56:03.018553 21609 raft_consensus.cc:374] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:03.018657 21609 raft_consensus.cc:729] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 385e80f2a3af46b58e98fe92c340dc96, State: Initialized, Role: FOLLOWER
I20260808 03:56:03.018474 21610 raft_consensus.cc:348] T 19a4c91e9f3347f2a302b3f1cbe2813f P b4009e96862048b79ecfd03ca827523d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eb5938cc474f7fbc798f0dafe6d902" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37177 } } peers { permanent_uuid: "385e80f2a3af46b58e98fe92c340dc96" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38123 } } peers { permanent_uuid: "b4009e96862048b79ecfd03ca827523d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 35237 } }
I20260808 03:56:03.018890 21610 raft_consensus.cc:374] T 19a4c91e9f3347f2a302b3f1cbe2813f P b4009e96862048b79ecfd03ca827523d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:03.018997 21610 raft_consensus.cc:729] T 19a4c91e9f3347f2a302b3f1cbe2813f P b4009e96862048b79ecfd03ca827523d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4009e96862048b79ecfd03ca827523d, State: Initialized, Role: FOLLOWER
I20260808 03:56:03.019416 21609 consensus_queue.cc:260] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eb5938cc474f7fbc798f0dafe6d902" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37177 } } peers { permanent_uuid: "385e80f2a3af46b58e98fe92c340dc96" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38123 } } peers { permanent_uuid: "b4009e96862048b79ecfd03ca827523d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 35237 } }
I20260808 03:56:03.019577 21610 consensus_queue.cc:260] T 19a4c91e9f3347f2a302b3f1cbe2813f P b4009e96862048b79ecfd03ca827523d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eb5938cc474f7fbc798f0dafe6d902" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37177 } } peers { permanent_uuid: "385e80f2a3af46b58e98fe92c340dc96" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38123 } } peers { permanent_uuid: "b4009e96862048b79ecfd03ca827523d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 35237 } }
I20260808 03:56:03.020502 21391 heartbeater.cc:503] Master 127.26.146.190:41557 was elected leader, sending a full tablet report...
I20260808 03:56:03.021226 21609 ts_tablet_manager.cc:1428] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96: Time spent starting tablet: real 0.012s user 0.009s sys 0.002s
I20260808 03:56:03.022145 21580 heartbeater.cc:503] Master 127.26.146.190:41557 was elected leader, sending a full tablet report...
I20260808 03:56:03.022737 21610 ts_tablet_manager.cc:1428] T 19a4c91e9f3347f2a302b3f1cbe2813f P b4009e96862048b79ecfd03ca827523d: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260808 03:56:03.090760 21616 raft_consensus.cc:482] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:56:03.091081 21616 raft_consensus.cc:504] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eb5938cc474f7fbc798f0dafe6d902" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37177 } } peers { permanent_uuid: "385e80f2a3af46b58e98fe92c340dc96" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38123 } } peers { permanent_uuid: "b4009e96862048b79ecfd03ca827523d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 35237 } }
I20260808 03:56:03.092828 21616 leader_election.cc:288] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8eb5938cc474f7fbc798f0dafe6d902 (127.26.146.129:37177), b4009e96862048b79ecfd03ca827523d (127.26.146.131:35237)
I20260808 03:56:03.099362 21601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:03.093431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:59075 (local address 127.26.146.129:37177)
0808 03:56:03.093826 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:03.093832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:03.094211 (+ 379us) server_negotiation.cc:407] Connection header received
0808 03:56:03.094390 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:03.094394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:03.094464 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:03.094536 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:03.095140 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:03.095851 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:03.097717 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:03.098494 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:03.098627 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:03.098706 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:03.098993 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 03:56:03.099110 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259}
I20260808 03:56:03.099370 21605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:03.093331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:57739 (local address 127.26.146.131:35237)
0808 03:56:03.093663 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:03.093670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:03.094079 (+ 409us) server_negotiation.cc:407] Connection header received
0808 03:56:03.094289 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:03.094293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:03.094352 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:03.094424 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:03.095191 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:03.095926 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:03.097760 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:03.098524 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:03.098671 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:03.098739 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:03.099023 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 03:56:03.099156 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182}
I20260808 03:56:03.099596 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:03.093087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:35237 (local address 127.26.146.130:57739)
0808 03:56:03.093956 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:03.093980 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:03.094139 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:03.094453 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:03.094458 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:03.094482 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:03.095047 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:03.095060 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:03.096054 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:03.096061 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:03.097578 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:03.097593 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:03.098711 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:03.098717 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:03.098869 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:03.098891 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 03:56:03.098967 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 03:56:03.099082 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"mutex_wait_us":1,"thread_start_us":473,"threads_started":1}
I20260808 03:56:03.099677 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:03.093114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:37177 (local address 127.26.146.130:59075)
0808 03:56:03.094108 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:03.094125 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:03.094254 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:03.094576 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:03.094580 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:03.094602 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:03.094995 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:03.095007 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:03.095976 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:03.095981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:03.097537 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:03.097549 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:03.098697 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:03.098703 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:03.098852 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:03.098876 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 03:56:03.098954 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 03:56:03.099075 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":874,"mutex_wait_us":73,"thread_start_us":725,"threads_started":1}
I20260808 03:56:03.100556 21509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19a4c91e9f3347f2a302b3f1cbe2813f" candidate_uuid: "385e80f2a3af46b58e98fe92c340dc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4009e96862048b79ecfd03ca827523d" is_pre_election: true
I20260808 03:56:03.100729 21139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19a4c91e9f3347f2a302b3f1cbe2813f" candidate_uuid: "385e80f2a3af46b58e98fe92c340dc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8eb5938cc474f7fbc798f0dafe6d902" is_pre_election: true
I20260808 03:56:03.101059 21509 raft_consensus.cc:2454] T 19a4c91e9f3347f2a302b3f1cbe2813f P b4009e96862048b79ecfd03ca827523d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 385e80f2a3af46b58e98fe92c340dc96 in term 0.
I20260808 03:56:03.101217 21139 raft_consensus.cc:2454] T 19a4c91e9f3347f2a302b3f1cbe2813f P a8eb5938cc474f7fbc798f0dafe6d902 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 385e80f2a3af46b58e98fe92c340dc96 in term 0.
I20260808 03:56:03.101680 21228 leader_election.cc:302] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 385e80f2a3af46b58e98fe92c340dc96, b4009e96862048b79ecfd03ca827523d; no voters:
I20260808 03:56:03.102037 21616 raft_consensus.cc:2781] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:56:03.102164 21616 raft_consensus.cc:482] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:56:03.102240 21616 raft_consensus.cc:3037] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:03.104470 21616 raft_consensus.cc:504] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eb5938cc474f7fbc798f0dafe6d902" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37177 } } peers { permanent_uuid: "385e80f2a3af46b58e98fe92c340dc96" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38123 } } peers { permanent_uuid: "b4009e96862048b79ecfd03ca827523d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 35237 } }
I20260808 03:56:03.105127 21616 leader_election.cc:288] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96 [CANDIDATE]: Term 1 election: Requested vote from peers a8eb5938cc474f7fbc798f0dafe6d902 (127.26.146.129:37177), b4009e96862048b79ecfd03ca827523d (127.26.146.131:35237)
I20260808 03:56:03.105458 21139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19a4c91e9f3347f2a302b3f1cbe2813f" candidate_uuid: "385e80f2a3af46b58e98fe92c340dc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8eb5938cc474f7fbc798f0dafe6d902"
I20260808 03:56:03.105628 21139 raft_consensus.cc:3037] T 19a4c91e9f3347f2a302b3f1cbe2813f P a8eb5938cc474f7fbc798f0dafe6d902 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:03.105612 21509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19a4c91e9f3347f2a302b3f1cbe2813f" candidate_uuid: "385e80f2a3af46b58e98fe92c340dc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4009e96862048b79ecfd03ca827523d"
I20260808 03:56:03.105830 21509 raft_consensus.cc:3037] T 19a4c91e9f3347f2a302b3f1cbe2813f P b4009e96862048b79ecfd03ca827523d [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:03.107930 21139 raft_consensus.cc:2454] T 19a4c91e9f3347f2a302b3f1cbe2813f P a8eb5938cc474f7fbc798f0dafe6d902 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 385e80f2a3af46b58e98fe92c340dc96 in term 1.
I20260808 03:56:03.108180 21509 raft_consensus.cc:2454] T 19a4c91e9f3347f2a302b3f1cbe2813f P b4009e96862048b79ecfd03ca827523d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 385e80f2a3af46b58e98fe92c340dc96 in term 1.
I20260808 03:56:03.108335 21230 leader_election.cc:302] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 385e80f2a3af46b58e98fe92c340dc96, a8eb5938cc474f7fbc798f0dafe6d902; no voters:
I20260808 03:56:03.108666 21616 raft_consensus.cc:2781] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:03.109081 21616 raft_consensus.cc:686] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96 [term 1 LEADER]: Becoming Leader. State: Replica: 385e80f2a3af46b58e98fe92c340dc96, State: Running, Role: LEADER
I20260808 03:56:03.109712 21616 consensus_queue.cc:237] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eb5938cc474f7fbc798f0dafe6d902" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37177 } } peers { permanent_uuid: "385e80f2a3af46b58e98fe92c340dc96" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38123 } } peers { permanent_uuid: "b4009e96862048b79ecfd03ca827523d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 35237 } }
I20260808 03:56:03.114567 20940 catalog_manager.cc:5495] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 385e80f2a3af46b58e98fe92c340dc96 (127.26.146.130). New cstate: current_term: 1 leader_uuid: "385e80f2a3af46b58e98fe92c340dc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eb5938cc474f7fbc798f0dafe6d902" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "385e80f2a3af46b58e98fe92c340dc96" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 38123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4009e96862048b79ecfd03ca827523d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 35237 } health_report { overall_health: UNKNOWN } } }
I20260808 03:56:03.131392 21603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:03.125985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42736 (local address 127.26.146.130:38123)
0808 03:56:03.126175 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:03.126181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:03.126287 (+ 106us) server_negotiation.cc:407] Connection header received
0808 03:56:03.126503 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:03.126507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:03.126636 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:03.126761 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:56:03.127131 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:03.127882 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:03.129334 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:03.129750 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:03.129859 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:03.130332 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:56:03.130831 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:56:03.130970 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:03.131052 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 03:56:03.131130 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 03:56:03.138963 21581 tablet.cc:2369] T 19a4c91e9f3347f2a302b3f1cbe2813f P b4009e96862048b79ecfd03ca827523d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:56:03.138999 21509 raft_consensus.cc:1261] T 19a4c91e9f3347f2a302b3f1cbe2813f P b4009e96862048b79ecfd03ca827523d [term 1 FOLLOWER]: Refusing update from remote peer 385e80f2a3af46b58e98fe92c340dc96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:56:03.139048 21139 raft_consensus.cc:1261] T 19a4c91e9f3347f2a302b3f1cbe2813f P a8eb5938cc474f7fbc798f0dafe6d902 [term 1 FOLLOWER]: Refusing update from remote peer 385e80f2a3af46b58e98fe92c340dc96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:56:03.140074 21620 consensus_queue.cc:1035] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4009e96862048b79ecfd03ca827523d" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 35237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:56:03.140398 21616 consensus_queue.cc:1035] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8eb5938cc474f7fbc798f0dafe6d902" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 37177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:56:03.151209 21627 mvcc.cc:205] Tried to move back new op lower bound from 7316116943408201728 to 7316116943301083136. Current Snapshot: MvccSnapshot[applied={T|T < 7316116943408201728}]
I20260808 03:56:03.168270 20940 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:56:03.168516 20940 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:56:03.171945 20940 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47087fa26a3f4c7593f8a4ec7b73ac7e: Sending DeleteTablet for 3 replicas of tablet 19a4c91e9f3347f2a302b3f1cbe2813f
I20260808 03:56:03.173012 21294 tablet_service.cc:1514] Processing DeleteTablet for tablet 19a4c91e9f3347f2a302b3f1cbe2813f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:56:03 UTC) from {username='slave'} at 127.0.0.1:42724
I20260808 03:56:03.173040 21110 tablet_service.cc:1514] Processing DeleteTablet for tablet 19a4c91e9f3347f2a302b3f1cbe2813f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:56:03 UTC) from {username='slave'} at 127.0.0.1:41310
I20260808 03:56:03.173648 21480 tablet_service.cc:1514] Processing DeleteTablet for tablet 19a4c91e9f3347f2a302b3f1cbe2813f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:56:03 UTC) from {username='slave'} at 127.0.0.1:59340
I20260808 03:56:03.175350 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 21027
I20260808 03:56:03.185267 21644 tablet_replica.cc:330] T 19a4c91e9f3347f2a302b3f1cbe2813f P b4009e96862048b79ecfd03ca827523d: stopping tablet replica
I20260808 03:56:03.185740 21644 raft_consensus.cc:2229] T 19a4c91e9f3347f2a302b3f1cbe2813f P b4009e96862048b79ecfd03ca827523d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:56:03.186252 21644 raft_consensus.cc:2258] T 19a4c91e9f3347f2a302b3f1cbe2813f P b4009e96862048b79ecfd03ca827523d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:56:03.189107 21644 ts_tablet_manager.cc:1905] T 19a4c91e9f3347f2a302b3f1cbe2813f P b4009e96862048b79ecfd03ca827523d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:56:03.198078 21644 ts_tablet_manager.cc:1918] T 19a4c91e9f3347f2a302b3f1cbe2813f P b4009e96862048b79ecfd03ca827523d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:56:03.198254 21644 log.cc:1196] T 19a4c91e9f3347f2a302b3f1cbe2813f P b4009e96862048b79ecfd03ca827523d: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/19a4c91e9f3347f2a302b3f1cbe2813f
I20260808 03:56:03.198834 21644 ts_tablet_manager.cc:1939] T 19a4c91e9f3347f2a302b3f1cbe2813f P b4009e96862048b79ecfd03ca827523d: Deleting consensus metadata
I20260808 03:56:03.199461 21601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:03.177074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41322 (local address 127.26.146.129:37177)
0808 03:56:03.177280 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:03.177287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:03.187047 (+ 9760us) server_negotiation.cc:407] Connection header received
0808 03:56:03.187386 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:03.187393 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:03.187495 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:03.187591 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:03.189317 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:03.190438 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:03.194113 (+ 3675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:03.194710 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:03.194812 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:03.197288 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:03.197300 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:03.197318 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:03.198081 (+ 763us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:03.198142 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:03.198169 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:03.198475 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:03.198885 (+ 410us) server_negotiation.cc:299] Negotiation successful
0808 03:56:03.199099 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 03:56:03.200320 21638 tablet_replica.cc:330] T 19a4c91e9f3347f2a302b3f1cbe2813f P a8eb5938cc474f7fbc798f0dafe6d902: stopping tablet replica
I20260808 03:56:03.200773 21638 raft_consensus.cc:2229] T 19a4c91e9f3347f2a302b3f1cbe2813f P a8eb5938cc474f7fbc798f0dafe6d902 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:56:03.201180 21199 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:03.205205 21639 tablet_replica.cc:330] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96: stopping tablet replica
I20260808 03:56:03.205657 21639 raft_consensus.cc:2229] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96 [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:56:03.206004 20919 catalog_manager.cc:4841] TS b4009e96862048b79ecfd03ca827523d (127.26.146.131:35237): tablet 19a4c91e9f3347f2a302b3f1cbe2813f (table test-table [id=868e37a98ad24d968f90a124e5f77936]) successfully deleted
I20260808 03:56:03.206440 21639 raft_consensus.cc:2258] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:56:03.208667 21639 ts_tablet_manager.cc:1905] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:56:03.216385 21639 ts_tablet_manager.cc:1918] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:56:03.216589 21639 log.cc:1196] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/19a4c91e9f3347f2a302b3f1cbe2813f
I20260808 03:56:03.217154 21639 ts_tablet_manager.cc:1939] T 19a4c91e9f3347f2a302b3f1cbe2813f P 385e80f2a3af46b58e98fe92c340dc96: Deleting consensus metadata
I20260808 03:56:03.218981 20920 catalog_manager.cc:4841] TS 385e80f2a3af46b58e98fe92c340dc96 (127.26.146.130:38123): tablet 19a4c91e9f3347f2a302b3f1cbe2813f (table test-table [id=868e37a98ad24d968f90a124e5f77936]) successfully deleted
I20260808 03:56:03.291960 21638 raft_consensus.cc:2258] T 19a4c91e9f3347f2a302b3f1cbe2813f P a8eb5938cc474f7fbc798f0dafe6d902 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:56:03.292703 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 21027
W20260808 03:56:03.323189 20922 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.146.129:37177 (error 108)
I20260808 03:56:03.323412 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 21211
I20260808 03:56:03.325310 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:03.324809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:37177 (local address 127.0.0.1:41334)
0808 03:56:03.324996 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:03.325133 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:37177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 03:56:03.325567 20922 catalog_manager.cc:4584] TS a8eb5938cc474f7fbc798f0dafe6d902 (127.26.146.129:37177): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 19a4c91e9f3347f2a302b3f1cbe2813f: Network error: Client connection negotiation failed: client connection to 127.26.146.129:37177: connect: Connection refused (error 111)
I20260808 03:56:03.331076 21603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:03.324519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42750 (local address 127.26.146.130:38123)
0808 03:56:03.324707 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:03.324713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:03.324734 (+ 21us) server_negotiation.cc:407] Connection header received
0808 03:56:03.324841 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:03.324847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:03.324955 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:03.325088 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:03.325582 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:03.326413 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:03.327680 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:03.328299 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:03.328422 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:03.329567 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:03.329576 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:03.329591 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:03.330221 (+ 630us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:03.330273 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:03.330293 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:03.330395 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:03.330610 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 03:56:03.330709 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 03:56:03.331768 21381 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:03.379092 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:03.378611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:37177 (local address 127.0.0.1:41348)
0808 03:56:03.378831 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:03.378933 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:37177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 03:56:03.380226 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:03.379874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:37177 (local address 127.0.0.1:41352)
0808 03:56:03.380010 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:03.380083 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:37177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 03:56:03.441004 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:03.440418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:37177 (local address 127.0.0.1:41368)
0808 03:56:03.440685 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:03.440795 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:37177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260808 03:56:03.442507 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 21211
I20260808 03:56:03.442553 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:03.442102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:37177 (local address 127.0.0.1:41382)
0808 03:56:03.442326 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:03.442421 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:37177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260808 03:56:03.464529 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 21397
I20260808 03:56:03.471118 21605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:03.465330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59354 (local address 127.26.146.131:35237)
0808 03:56:03.465504 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:03.465508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:03.465527 (+ 19us) server_negotiation.cc:407] Connection header received
0808 03:56:03.465685 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:03.465689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:03.465771 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:03.465849 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:03.466191 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:03.466923 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:03.468196 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:03.468571 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:03.468658 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:03.469847 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:03.469860 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:03.469882 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:03.470453 (+ 571us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:03.470501 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:03.470522 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:03.470605 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:03.470738 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 03:56:03.470831 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 03:56:03.471792 21572 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:03.523355 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:03.522784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:37177 (local address 127.0.0.1:41392)
0808 03:56:03.523045 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:03.523165 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:37177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260808 03:56:03.524593 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:03.524137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:37177 (local address 127.0.0.1:41408)
0808 03:56:03.524360 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:03.524453 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:37177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260808 03:56:03.558079 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 21397
I20260808 03:56:03.587316 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 20900
I20260808 03:56:03.588609 20988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:03.722656 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:03.721972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:37177 (local address 127.0.0.1:41420)
0808 03:56:03.722260 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:03.722394 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:37177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260808 03:56:03.722743 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 20900
2026-08-08T03:56:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2815 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:56:03 dist-test-slave-jlzn krb5kdc[21653](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:56:03 dist-test-slave-jlzn krb5kdc[21653](info): set up 2 sockets
Aug 08 03:56:03 dist-test-slave-jlzn krb5kdc[21653](info): commencing operation
krb5kdc: starting...
W20260808 03:56:04.934603 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:56:05 dist-test-slave-jlzn krb5kdc[21653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:56:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:56:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:56:05.211205 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:45029
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:37365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:45029
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:05.540586 21674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:05.540895 21674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:05.541028 21674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:05.548085 21674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:05.548197 21674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:05.548274 21674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:05.548347 21674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:05.548406 21674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:05.557961 21674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:37365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:45029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:45029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:05.559270 21674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:05.560575 21674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:05.571938 21683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:05.572006 21680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:05.572038 21679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:05.572906 21674 server_base.cc:1034] running on GCE node
I20260808 03:56:05.573702 21674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:05.575048 21674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:05.576241 21674 hybrid_clock.cc:648] HybridClock initialized: now 1786161365576181 us; error 47 us; skew 500 ppm
I20260808 03:56:05.576574 21674 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:05 dist-test-slave-jlzn krb5kdc[21653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161365, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:05.579531 21674 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:56:05.580919 21674 webserver.cc:460] Webserver started at http://127.26.146.190:39099/ using document root <none> and password file <none>
I20260808 03:56:05.581558 21674 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:05.581681 21674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:05.581970 21674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:05.584915 21674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "4f166c8e5dfb47c6a0525522363ba0de"
format_stamp: "Formatted at 2026-08-08 03:56:05 on dist-test-slave-jlzn"
server_key: "c87c660f1fa5942e2712c5489b16e99c"
server_key_iv: "bb2c53edc8c2b80810b2535da5e0276b"
server_key_version: "encryptionkey@0"
I20260808 03:56:05.585657 21674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "4f166c8e5dfb47c6a0525522363ba0de"
format_stamp: "Formatted at 2026-08-08 03:56:05 on dist-test-slave-jlzn"
server_key: "c87c660f1fa5942e2712c5489b16e99c"
server_key_iv: "bb2c53edc8c2b80810b2535da5e0276b"
server_key_version: "encryptionkey@0"
I20260808 03:56:05.590744 21674 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 03:56:05.594205 21694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:05.595789 21674 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 03:56:05.595996 21674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "4f166c8e5dfb47c6a0525522363ba0de"
format_stamp: "Formatted at 2026-08-08 03:56:05 on dist-test-slave-jlzn"
server_key: "c87c660f1fa5942e2712c5489b16e99c"
server_key_iv: "bb2c53edc8c2b80810b2535da5e0276b"
server_key_version: "encryptionkey@0"
I20260808 03:56:05.596182 21674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:05.610621 21674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:05.614547 21674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:05.614804 21674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:05.632928 21674 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:45029
I20260808 03:56:05.632948 21769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:45029 every 8 connection(s)
I20260808 03:56:05.634313 21674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:56:05.637256 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 21674
I20260808 03:56:05.637454 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:56:05.638080 27210 external_mini_cluster.cc:1428] Setting key e2564c25358fbe040d38ef62b13cc3b6
I20260808 03:56:05.638484 21770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:05.650487 21770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f166c8e5dfb47c6a0525522363ba0de: Bootstrap starting.
Aug 08 03:56:05 dist-test-slave-jlzn krb5kdc[21653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:56:05.654006 21770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f166c8e5dfb47c6a0525522363ba0de: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:05.655157 21770 log.cc:824] T 00000000000000000000000000000000 P 4f166c8e5dfb47c6a0525522363ba0de: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:05.658376 21770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f166c8e5dfb47c6a0525522363ba0de: No bootstrap required, opened a new log
I20260808 03:56:05.660444 21773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:05.639984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55510 (local address 127.26.146.190:45029)
0808 03:56:05.640710 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:05.640721 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:05.640761 (+ 40us) server_negotiation.cc:407] Connection header received
0808 03:56:05.643033 (+ 2272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:05.643056 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:05.643404 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:05.643753 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:05.645259 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:05.646510 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:05.647846 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:05.648532 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:05.648713 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:05.653186 (+ 4473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:05.653194 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:05.653209 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:05.653243 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:05.656509 (+ 3266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:05.657340 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:05.657346 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:05.657351 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:05.657511 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:05.657909 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:05.657912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:05.657914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:05.658397 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:05.658664 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:05.659108 (+ 444us) server_negotiation.cc:299] Negotiation successful
0808 03:56:05.659409 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":303,"threads_started":1}
I20260808 03:56:05.666446 21770 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f166c8e5dfb47c6a0525522363ba0de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f166c8e5dfb47c6a0525522363ba0de" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 45029 } }
I20260808 03:56:05.666778 21770 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f166c8e5dfb47c6a0525522363ba0de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:05.666879 21770 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f166c8e5dfb47c6a0525522363ba0de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f166c8e5dfb47c6a0525522363ba0de, State: Initialized, Role: FOLLOWER
I20260808 03:56:05.667392 21770 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f166c8e5dfb47c6a0525522363ba0de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f166c8e5dfb47c6a0525522363ba0de" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 45029 } }
I20260808 03:56:05.667634 21770 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f166c8e5dfb47c6a0525522363ba0de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:56:05.667769 21770 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f166c8e5dfb47c6a0525522363ba0de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:56:05.667953 21770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f166c8e5dfb47c6a0525522363ba0de [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:05.670080 21770 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f166c8e5dfb47c6a0525522363ba0de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f166c8e5dfb47c6a0525522363ba0de" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 45029 } }
I20260808 03:56:05.670542 21770 leader_election.cc:302] T 00000000000000000000000000000000 P 4f166c8e5dfb47c6a0525522363ba0de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f166c8e5dfb47c6a0525522363ba0de; no voters:
I20260808 03:56:05.671114 21770 leader_election.cc:288] T 00000000000000000000000000000000 P 4f166c8e5dfb47c6a0525522363ba0de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:56:05.671227 21775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f166c8e5dfb47c6a0525522363ba0de [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:05.671869 21775 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f166c8e5dfb47c6a0525522363ba0de [term 1 LEADER]: Becoming Leader. State: Replica: 4f166c8e5dfb47c6a0525522363ba0de, State: Running, Role: LEADER
I20260808 03:56:05.672375 21775 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f166c8e5dfb47c6a0525522363ba0de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f166c8e5dfb47c6a0525522363ba0de" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 45029 } }
I20260808 03:56:05.673059 21770 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f166c8e5dfb47c6a0525522363ba0de [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:05.674799 21776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f166c8e5dfb47c6a0525522363ba0de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f166c8e5dfb47c6a0525522363ba0de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f166c8e5dfb47c6a0525522363ba0de" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 45029 } } }
I20260808 03:56:05.675201 21776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f166c8e5dfb47c6a0525522363ba0de [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:05.679975 21777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f166c8e5dfb47c6a0525522363ba0de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f166c8e5dfb47c6a0525522363ba0de. Latest consensus state: current_term: 1 leader_uuid: "4f166c8e5dfb47c6a0525522363ba0de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f166c8e5dfb47c6a0525522363ba0de" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 45029 } } }
I20260808 03:56:05.680249 21777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f166c8e5dfb47c6a0525522363ba0de [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:05.687872 21787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:56:05.692046 21787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:56:05.699901 21787 catalog_manager.cc:1347] Generated new cluster ID: 9b1e11dbe2b1446698b1390db0e11c44
I20260808 03:56:05.699995 21787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:56:05.722925 21787 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:56:05.724316 21787 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:56:05.738246 21787 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f166c8e5dfb47c6a0525522363ba0de: Generated new TSK 0
I20260808 03:56:05.739156 21787 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
I20260808 03:56:05.820927 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:45029
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:37365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:06.185600 21806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:06.185870 21806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:06.185986 21806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:06.193074 21806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:06.193178 21806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:06.193329 21806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:56:06.202894 21806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:37365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:45029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:06.204162 21806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:06.205420 21806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:06.217918 21813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:06.218021 21819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:06.218000 21821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:06.218923 21806 server_base.cc:1034] running on GCE node
I20260808 03:56:06.219496 21806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:06.220463 21806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:06.221660 21806 hybrid_clock.cc:648] HybridClock initialized: now 1786161366221619 us; error 32 us; skew 500 ppm
I20260808 03:56:06.221990 21806 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:06 dist-test-slave-jlzn krb5kdc[21653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161366, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:06.225011 21806 init.cc:377] Logged in from keytab as kudu/127.26.146.129@KRBTEST.COM (short username kudu)
I20260808 03:56:06.226418 21806 webserver.cc:460] Webserver started at http://127.26.146.129:32785/ using document root <none> and password file <none>
I20260808 03:56:06.227066 21806 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:06.227185 21806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:06.227473 21806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:06.230302 21806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "52cb109847d5481f9c3c79877b68fdcd"
format_stamp: "Formatted at 2026-08-08 03:56:06 on dist-test-slave-jlzn"
server_key: "c7e42efb4e87c1ceab69623de6d2b268"
server_key_iv: "62ecc53f6a1508445dd91c30d2711087"
server_key_version: "encryptionkey@0"
I20260808 03:56:06.231045 21806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "52cb109847d5481f9c3c79877b68fdcd"
format_stamp: "Formatted at 2026-08-08 03:56:06 on dist-test-slave-jlzn"
server_key: "c7e42efb4e87c1ceab69623de6d2b268"
server_key_iv: "62ecc53f6a1508445dd91c30d2711087"
server_key_version: "encryptionkey@0"
I20260808 03:56:06.235956 21806 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260808 03:56:06.239229 21830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:06.240583 21806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:56:06.240726 21806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "52cb109847d5481f9c3c79877b68fdcd"
format_stamp: "Formatted at 2026-08-08 03:56:06 on dist-test-slave-jlzn"
server_key: "c7e42efb4e87c1ceab69623de6d2b268"
server_key_iv: "62ecc53f6a1508445dd91c30d2711087"
server_key_version: "encryptionkey@0"
I20260808 03:56:06.240899 21806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:06.260686 21806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:06.264492 21806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:06.264768 21806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:06.265688 21806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:56:06.267020 21806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:56:06.267098 21806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:06.267210 21806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:56:06.267261 21806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:06.297154 21806 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:42729
I20260808 03:56:06.297202 22011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:42729 every 8 connection(s)
I20260808 03:56:06.298367 21806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:56:06.304713 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 21806
I20260808 03:56:06.304869 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:56:06.305325 27210 external_mini_cluster.cc:1428] Setting key edce04d164adebe481434817ccf89842
Aug 08 03:56:06 dist-test-slave-jlzn krb5kdc[21653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161366, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:56:06.318605 22014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:06.300856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:45673 (local address 127.26.146.190:45029)
0808 03:56:06.301316 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:06.301320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:06.302296 (+ 976us) server_negotiation.cc:407] Connection header received
0808 03:56:06.304217 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:06.304223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:06.304310 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:06.304407 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:06.305988 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:06.306756 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:06.308354 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:06.308742 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:06.308858 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:06.312425 (+ 3567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:06.312431 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:06.312435 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:06.312473 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:06.315530 (+ 3057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:06.316303 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:06.316308 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:06.316311 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:06.316458 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:06.316913 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:06.316917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:06.316920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:06.317278 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:06.317412 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:06.318049 (+ 637us) server_negotiation.cc:299] Negotiation successful
0808 03:56:06.318258 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":258,"threads_started":1}
I20260808 03:56:06.319406 22015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:06.301364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:45029 (local address 127.26.146.129:45673)
0808 03:56:06.302115 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:06.302163 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:06.303944 (+ 1781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:06.304667 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:06.304676 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:06.305224 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:06.305810 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:06.305828 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:06.306921 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:06.306925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:06.308208 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:06.308221 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:06.308987 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:06.308994 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:06.309210 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:06.309846 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:56:06.309872 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:06.312213 (+ 2341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:06.315669 (+ 3456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:06.315675 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:06.315688 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:06.316179 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:06.316600 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:06.316604 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:06.316607 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:06.316801 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:06.317389 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:06.317394 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:06.317690 (+ 296us) client_negotiation.cc:769] Sending connection context
0808 03:56:06.317904 (+ 214us) client_negotiation.cc:240] Negotiation successful
0808 03:56:06.318300 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":299,"threads_started":1}
I20260808 03:56:06.320983 22012 heartbeater.cc:347] Connected to a master server at 127.26.146.190:45029
I20260808 03:56:06.321305 22012 heartbeater.cc:464] Registering TS with master...
I20260808 03:56:06.322013 22012 heartbeater.cc:511] Master 127.26.146.190:45029 requested a full tablet report, sending...
I20260808 03:56:06.324362 21723 ts_manager.cc:194] Registered new tserver with Master: 52cb109847d5481f9c3c79877b68fdcd (127.26.146.129:42729)
I20260808 03:56:06.326305 21723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.129@KRBTEST.COM'} at 127.26.146.129:45673
WARNING: no policy specified for kudu/127.26.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
I20260808 03:56:06.370712 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:45029
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:37365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:06.701705 22019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:06.701959 22019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:06.702064 22019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:06.709398 22019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:06.709517 22019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:06.709637 22019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:56:06.719177 22019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:37365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:45029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:06.720404 22019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:06.721657 22019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:06.734560 22025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:06.734632 22031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:06.734606 22029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:06.735376 22019 server_base.cc:1034] running on GCE node
I20260808 03:56:06.735967 22019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:06.736848 22019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:06.738071 22019 hybrid_clock.cc:648] HybridClock initialized: now 1786161366738019 us; error 41 us; skew 500 ppm
I20260808 03:56:06.738395 22019 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:06 dist-test-slave-jlzn krb5kdc[21653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161366, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:06.744664 22019 init.cc:377] Logged in from keytab as kudu/127.26.146.130@KRBTEST.COM (short username kudu)
I20260808 03:56:06.746192 22019 webserver.cc:460] Webserver started at http://127.26.146.130:42751/ using document root <none> and password file <none>
I20260808 03:56:06.746851 22019 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:06.746975 22019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:06.747293 22019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:06.750483 22019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "c51c0a335cef4b3c8952da5cc307bbd0"
format_stamp: "Formatted at 2026-08-08 03:56:06 on dist-test-slave-jlzn"
server_key: "8204203e3bff094aa53c4f04b1585bc2"
server_key_iv: "2c30bb735063ffcf4dfe80bf63d58e9b"
server_key_version: "encryptionkey@0"
I20260808 03:56:06.751245 22019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "c51c0a335cef4b3c8952da5cc307bbd0"
format_stamp: "Formatted at 2026-08-08 03:56:06 on dist-test-slave-jlzn"
server_key: "8204203e3bff094aa53c4f04b1585bc2"
server_key_iv: "2c30bb735063ffcf4dfe80bf63d58e9b"
server_key_version: "encryptionkey@0"
I20260808 03:56:06.756263 22019 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260808 03:56:06.759505 22041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:06.760846 22019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:56:06.761010 22019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "c51c0a335cef4b3c8952da5cc307bbd0"
format_stamp: "Formatted at 2026-08-08 03:56:06 on dist-test-slave-jlzn"
server_key: "8204203e3bff094aa53c4f04b1585bc2"
server_key_iv: "2c30bb735063ffcf4dfe80bf63d58e9b"
server_key_version: "encryptionkey@0"
I20260808 03:56:06.761195 22019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:06.782959 22019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:06.786815 22019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:06.787092 22019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:06.788053 22019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:56:06.789351 22019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:56:06.789435 22019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:06.789565 22019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:56:06.789624 22019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:06.818871 22019 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:45823
I20260808 03:56:06.818902 22207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:45823 every 8 connection(s)
I20260808 03:56:06.820305 22019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:56:06.828210 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 22019
I20260808 03:56:06.828395 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:56:06.828962 27210 external_mini_cluster.cc:1428] Setting key a82e0a1411d523608f16652e9b7271e8
Aug 08 03:56:06 dist-test-slave-jlzn krb5kdc[21653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161366, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:56:06.840274 22211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:06.822757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:50377 (local address 127.26.146.190:45029)
0808 03:56:06.823167 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:06.823172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:06.824209 (+ 1037us) server_negotiation.cc:407] Connection header received
0808 03:56:06.826016 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:06.826022 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:06.826101 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:06.826203 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:06.827657 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:06.828642 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:06.830373 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:06.830975 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:06.831078 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:06.834923 (+ 3845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:06.834928 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:06.834931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:06.834962 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:06.837358 (+ 2396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:06.838033 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:06.838038 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:06.838040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:06.838166 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:06.838563 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:06.838568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:06.838571 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:06.838848 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:06.838976 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:06.839671 (+ 695us) server_negotiation.cc:299] Negotiation successful
0808 03:56:06.839917 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":196,"threads_started":1}
I20260808 03:56:06.841223 22212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:06.823328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:45029 (local address 127.26.146.130:50377)
0808 03:56:06.824054 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:06.824092 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:06.825761 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:06.826441 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:06.826449 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:06.826953 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:06.827484 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:06.827500 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:06.828793 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:06.828798 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:06.830230 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:06.830243 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:06.831199 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:06.831207 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:06.831509 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:06.832225 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:56:06.832276 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:06.834710 (+ 2434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:06.837487 (+ 2777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:06.837496 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:06.837512 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:06.837924 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:06.838268 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:06.838273 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:06.838277 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:06.838460 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:06.838970 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:06.838978 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:06.839308 (+ 330us) client_negotiation.cc:769] Sending connection context
0808 03:56:06.839544 (+ 236us) client_negotiation.cc:240] Negotiation successful
0808 03:56:06.840100 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":258,"threads_started":1}
I20260808 03:56:06.842552 22208 heartbeater.cc:347] Connected to a master server at 127.26.146.190:45029
I20260808 03:56:06.842844 22208 heartbeater.cc:464] Registering TS with master...
I20260808 03:56:06.843516 22208 heartbeater.cc:511] Master 127.26.146.190:45029 requested a full tablet report, sending...
I20260808 03:56:06.844949 21723 ts_manager.cc:194] Registered new tserver with Master: c51c0a335cef4b3c8952da5cc307bbd0 (127.26.146.130:45823)
I20260808 03:56:06.845819 21723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.130@KRBTEST.COM'} at 127.26.146.130:50377
WARNING: no policy specified for kudu/127.26.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
I20260808 03:56:06.893661 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:45029
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:37365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:07.218158 22215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:07.218432 22215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:07.218567 22215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:07.225585 22215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:07.225704 22215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:07.225847 22215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:56:07.235181 22215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:37365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:45029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:07.236411 22215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:07.237690 22215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:07.250263 22221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:07.250236 22223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:07.250368 22225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:07.250885 22215 server_base.cc:1034] running on GCE node
I20260808 03:56:07.251470 22215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:07.252409 22215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:07.253623 22215 hybrid_clock.cc:648] HybridClock initialized: now 1786161367253566 us; error 49 us; skew 500 ppm
I20260808 03:56:07.253921 22215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:07 dist-test-slave-jlzn krb5kdc[21653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161367, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:07.257021 22215 init.cc:377] Logged in from keytab as kudu/127.26.146.131@KRBTEST.COM (short username kudu)
I20260808 03:56:07.258404 22215 webserver.cc:460] Webserver started at http://127.26.146.131:42305/ using document root <none> and password file <none>
I20260808 03:56:07.259059 22215 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:07.259176 22215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:07.259472 22215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:07.262296 22215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "6756602d01cc40708be44ba8e55ed744"
format_stamp: "Formatted at 2026-08-08 03:56:07 on dist-test-slave-jlzn"
server_key: "72158295976dee8fca3e0c1c50d8f3a8"
server_key_iv: "84f432e5b841145640c0879c02510bea"
server_key_version: "encryptionkey@0"
I20260808 03:56:07.263020 22215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "6756602d01cc40708be44ba8e55ed744"
format_stamp: "Formatted at 2026-08-08 03:56:07 on dist-test-slave-jlzn"
server_key: "72158295976dee8fca3e0c1c50d8f3a8"
server_key_iv: "84f432e5b841145640c0879c02510bea"
server_key_version: "encryptionkey@0"
I20260808 03:56:07.268100 22215 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 03:56:07.271464 22236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:07.272701 22215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:56:07.272835 22215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "6756602d01cc40708be44ba8e55ed744"
format_stamp: "Formatted at 2026-08-08 03:56:07 on dist-test-slave-jlzn"
server_key: "72158295976dee8fca3e0c1c50d8f3a8"
server_key_iv: "84f432e5b841145640c0879c02510bea"
server_key_version: "encryptionkey@0"
I20260808 03:56:07.273003 22215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:07.294349 22215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:07.298252 22215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:07.298509 22215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:07.299430 22215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:56:07.300839 22215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:56:07.300918 22215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:07.301035 22215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:56:07.301086 22215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:07.329346 22012 heartbeater.cc:503] Master 127.26.146.190:45029 was elected leader, sending a full tablet report...
I20260808 03:56:07.331638 22215 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:37155
I20260808 03:56:07.331717 22392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:37155 every 8 connection(s)
I20260808 03:56:07.332892 22215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:56:07.341001 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 22215
I20260808 03:56:07.341274 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:56:07.341832 27210 external_mini_cluster.cc:1428] Setting key 583fa8bfbd47c4a5e01426367af2d982
Aug 08 03:56:07 dist-test-slave-jlzn krb5kdc[21653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161367, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:56:07.352545 22211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:07.335240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:38309 (local address 127.26.146.190:45029)
0808 03:56:07.335425 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:07.335430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:07.336666 (+ 1236us) server_negotiation.cc:407] Connection header received
0808 03:56:07.338383 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:07.338387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:07.338453 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:07.338536 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:07.339968 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.340683 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.342810 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.343402 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.343531 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:07.347000 (+ 3469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:07.347011 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:07.347015 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:07.347046 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:07.349628 (+ 2582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:07.350311 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:07.350314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:07.350317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:07.350432 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:07.350837 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:07.350840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:07.350842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:07.351135 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:07.351278 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:07.351997 (+ 719us) server_negotiation.cc:299] Negotiation successful
0808 03:56:07.352220 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 03:56:07.353376 22398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:07.335781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:45029 (local address 127.26.146.131:38309)
0808 03:56:07.336482 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:07.336528 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:07.338158 (+ 1630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:07.338758 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:07.338766 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:07.339250 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:07.339831 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:07.339858 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.340830 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.340836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:07.342644 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:07.342658 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.343673 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.343682 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:07.343948 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:07.344633 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:56:07.344657 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:07.346791 (+ 2134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:07.349752 (+ 2961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:07.349761 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:07.349775 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:07.350192 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:07.350531 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:07.350536 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:07.350540 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:07.350723 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:07.351269 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:07.351277 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:07.351603 (+ 326us) client_negotiation.cc:769] Sending connection context
0808 03:56:07.351850 (+ 247us) client_negotiation.cc:240] Negotiation successful
0808 03:56:07.352344 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":248,"threads_started":1}
I20260808 03:56:07.354592 22394 heartbeater.cc:347] Connected to a master server at 127.26.146.190:45029
I20260808 03:56:07.354887 22394 heartbeater.cc:464] Registering TS with master...
I20260808 03:56:07.355681 22394 heartbeater.cc:511] Master 127.26.146.190:45029 requested a full tablet report, sending...
I20260808 03:56:07.357203 21723 ts_manager.cc:194] Registered new tserver with Master: 6756602d01cc40708be44ba8e55ed744 (127.26.146.131:37155)
I20260808 03:56:07.358006 21723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.131@KRBTEST.COM'} at 127.26.146.131:38309
I20260808 03:56:07.369053 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:56:07.385691 22211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:07.373146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55512 (local address 127.26.146.190:45029)
0808 03:56:07.373281 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:07.373285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:07.373719 (+ 434us) server_negotiation.cc:407] Connection header received
0808 03:56:07.373846 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:07.373849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:07.373913 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:07.373988 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:07.375291 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.376063 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.377500 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.377915 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.378012 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:07.380689 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:07.380694 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:07.380697 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:07.380725 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:07.383164 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:07.383935 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:07.383939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:07.383941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:07.384067 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:07.384494 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:07.384497 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:07.384499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:07.384777 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:07.384981 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:07.385314 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 03:56:07.385440 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 03:56:07.389532 21723 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:56:07.396308 21723 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:56:07.418195 22412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:07.407640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:45823 (local address 127.0.0.1:48080)
0808 03:56:07.408660 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:07.408703 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:07.408891 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:07.410113 (+ 1222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:07.410122 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:07.410155 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:07.410581 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:07.410594 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.412816 (+ 2222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.412822 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:07.415236 (+ 2414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:07.415254 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.417217 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.417223 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:07.417440 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:07.417495 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 03:56:07.417602 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 03:56:07.417793 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"thread_start_us":585,"threads_started":1}
I20260808 03:56:07.420091 22413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:07.407891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48080 (local address 127.26.146.130:45823)
0808 03:56:07.409440 (+ 1549us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:07.409449 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:07.409479 (+ 30us) server_negotiation.cc:407] Connection header received
0808 03:56:07.409591 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:07.409599 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:07.409833 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:07.410819 (+ 986us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:07.411034 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.412658 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.415455 (+ 2797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.416967 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.419274 (+ 2307us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:07.419462 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:07.419558 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 03:56:07.419729 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1399,"thread_start_us":1300,"threads_started":1}
I20260808 03:56:07.424619 22114 tablet_service.cc:1467] Processing CreateTablet for tablet 8ff6cc42c67b492d9e4e787b1eebef96 (DEFAULT_TABLE table=test-table [id=26f269a53fc6447dab23b68d0f087e5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:56:07.426203 22114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ff6cc42c67b492d9e4e787b1eebef96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:07.430469 22412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:07.407898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:42729 (local address 127.0.0.1:60150)
0808 03:56:07.418438 (+ 10540us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:07.418458 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:07.418556 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:07.424617 (+ 6061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:07.424622 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:07.424648 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:07.424831 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:07.424840 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.426574 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.426579 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:07.428200 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:07.428212 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.429599 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.429604 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:07.429856 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:07.429892 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 03:56:07.429977 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 03:56:07.430138 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10443,"mutex_wait_us":85,"thread_start_us":14181,"threads_started":1}
I20260808 03:56:07.430591 22418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:07.409682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55966 (local address 127.26.146.131:37155)
0808 03:56:07.421899 (+ 12217us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:07.421909 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:07.422604 (+ 695us) server_negotiation.cc:407] Connection header received
0808 03:56:07.422711 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:07.422719 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:07.422974 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:07.423172 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:07.424140 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.425360 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.427594 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.428961 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.429149 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:07.429314 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:07.430087 (+ 773us) server_negotiation.cc:299] Negotiation successful
0808 03:56:07.430249 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12050,"thread_start_us":11889,"threads_started":1}
I20260808 03:56:07.431212 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:07.408666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:37155 (local address 127.0.0.1:55966)
0808 03:56:07.422331 (+ 13665us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:07.422349 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:07.422469 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:07.423480 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:07.423486 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:07.423513 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:07.423928 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:07.423941 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.425572 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.425576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:07.427429 (+ 1853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:07.427445 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.429555 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.429564 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:07.429879 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:07.429905 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 03:56:07.429977 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 03:56:07.430861 (+ 884us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13549,"mutex_wait_us":842,"thread_start_us":15342,"threads_started":1}
I20260808 03:56:07.431578 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:07.409405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60150 (local address 127.26.146.129:42729)
0808 03:56:07.424068 (+ 14663us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:07.424075 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:07.424103 (+ 28us) server_negotiation.cc:407] Connection header received
0808 03:56:07.424197 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:07.424204 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:07.424414 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:07.424610 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:07.424971 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.426398 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.428344 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.429419 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.430852 (+ 1433us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:07.431030 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:07.431105 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 03:56:07.431258 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14500,"thread_start_us":14341,"threads_started":1}
I20260808 03:56:07.434991 22307 tablet_service.cc:1467] Processing CreateTablet for tablet 8ff6cc42c67b492d9e4e787b1eebef96 (DEFAULT_TABLE table=test-table [id=26f269a53fc6447dab23b68d0f087e5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:56:07.435072 21917 tablet_service.cc:1467] Processing CreateTablet for tablet 8ff6cc42c67b492d9e4e787b1eebef96 (DEFAULT_TABLE table=test-table [id=26f269a53fc6447dab23b68d0f087e5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:56:07.436259 22307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ff6cc42c67b492d9e4e787b1eebef96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:07.436473 21917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ff6cc42c67b492d9e4e787b1eebef96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:07.440404 22419 tablet_bootstrap.cc:492] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0: Bootstrap starting.
I20260808 03:56:07.444033 22419 tablet_bootstrap.cc:654] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:07.445400 22419 log.cc:824] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:07.448489 22422 tablet_bootstrap.cc:492] T 8ff6cc42c67b492d9e4e787b1eebef96 P 52cb109847d5481f9c3c79877b68fdcd: Bootstrap starting.
I20260808 03:56:07.448561 22419 tablet_bootstrap.cc:492] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0: No bootstrap required, opened a new log
I20260808 03:56:07.448928 22419 ts_tablet_manager.cc:1397] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260808 03:56:07.450085 22424 tablet_bootstrap.cc:492] T 8ff6cc42c67b492d9e4e787b1eebef96 P 6756602d01cc40708be44ba8e55ed744: Bootstrap starting.
I20260808 03:56:07.452046 22422 tablet_bootstrap.cc:654] T 8ff6cc42c67b492d9e4e787b1eebef96 P 52cb109847d5481f9c3c79877b68fdcd: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:07.453352 22424 tablet_bootstrap.cc:654] T 8ff6cc42c67b492d9e4e787b1eebef96 P 6756602d01cc40708be44ba8e55ed744: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:07.453323 22422 log.cc:824] T 8ff6cc42c67b492d9e4e787b1eebef96 P 52cb109847d5481f9c3c79877b68fdcd: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:07.454401 22424 log.cc:824] T 8ff6cc42c67b492d9e4e787b1eebef96 P 6756602d01cc40708be44ba8e55ed744: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:07.456207 22422 tablet_bootstrap.cc:492] T 8ff6cc42c67b492d9e4e787b1eebef96 P 52cb109847d5481f9c3c79877b68fdcd: No bootstrap required, opened a new log
I20260808 03:56:07.456470 22422 ts_tablet_manager.cc:1397] T 8ff6cc42c67b492d9e4e787b1eebef96 P 52cb109847d5481f9c3c79877b68fdcd: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260808 03:56:07.457278 22424 tablet_bootstrap.cc:492] T 8ff6cc42c67b492d9e4e787b1eebef96 P 6756602d01cc40708be44ba8e55ed744: No bootstrap required, opened a new log
I20260808 03:56:07.457517 22424 ts_tablet_manager.cc:1397] T 8ff6cc42c67b492d9e4e787b1eebef96 P 6756602d01cc40708be44ba8e55ed744: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260808 03:56:07.457346 22419 raft_consensus.cc:348] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c51c0a335cef4b3c8952da5cc307bbd0" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45823 } } peers { permanent_uuid: "52cb109847d5481f9c3c79877b68fdcd" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 42729 } } peers { permanent_uuid: "6756602d01cc40708be44ba8e55ed744" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37155 } }
I20260808 03:56:07.457741 22419 raft_consensus.cc:374] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:07.457844 22419 raft_consensus.cc:729] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c51c0a335cef4b3c8952da5cc307bbd0, State: Initialized, Role: FOLLOWER
I20260808 03:56:07.458392 22419 consensus_queue.cc:260] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c51c0a335cef4b3c8952da5cc307bbd0" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45823 } } peers { permanent_uuid: "52cb109847d5481f9c3c79877b68fdcd" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 42729 } } peers { permanent_uuid: "6756602d01cc40708be44ba8e55ed744" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37155 } }
I20260808 03:56:07.459451 22208 heartbeater.cc:503] Master 127.26.146.190:45029 was elected leader, sending a full tablet report...
I20260808 03:56:07.459998 22419 ts_tablet_manager.cc:1428] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0: Time spent starting tablet: real 0.011s user 0.010s sys 0.001s
I20260808 03:56:07.465927 22422 raft_consensus.cc:348] T 8ff6cc42c67b492d9e4e787b1eebef96 P 52cb109847d5481f9c3c79877b68fdcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c51c0a335cef4b3c8952da5cc307bbd0" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45823 } } peers { permanent_uuid: "52cb109847d5481f9c3c79877b68fdcd" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 42729 } } peers { permanent_uuid: "6756602d01cc40708be44ba8e55ed744" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37155 } }
I20260808 03:56:07.466316 22422 raft_consensus.cc:374] T 8ff6cc42c67b492d9e4e787b1eebef96 P 52cb109847d5481f9c3c79877b68fdcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:07.466406 22422 raft_consensus.cc:729] T 8ff6cc42c67b492d9e4e787b1eebef96 P 52cb109847d5481f9c3c79877b68fdcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52cb109847d5481f9c3c79877b68fdcd, State: Initialized, Role: FOLLOWER
I20260808 03:56:07.466400 22424 raft_consensus.cc:348] T 8ff6cc42c67b492d9e4e787b1eebef96 P 6756602d01cc40708be44ba8e55ed744 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c51c0a335cef4b3c8952da5cc307bbd0" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45823 } } peers { permanent_uuid: "52cb109847d5481f9c3c79877b68fdcd" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 42729 } } peers { permanent_uuid: "6756602d01cc40708be44ba8e55ed744" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37155 } }
I20260808 03:56:07.466763 22424 raft_consensus.cc:374] T 8ff6cc42c67b492d9e4e787b1eebef96 P 6756602d01cc40708be44ba8e55ed744 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:07.466858 22424 raft_consensus.cc:729] T 8ff6cc42c67b492d9e4e787b1eebef96 P 6756602d01cc40708be44ba8e55ed744 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6756602d01cc40708be44ba8e55ed744, State: Initialized, Role: FOLLOWER
I20260808 03:56:07.466972 22422 consensus_queue.cc:260] T 8ff6cc42c67b492d9e4e787b1eebef96 P 52cb109847d5481f9c3c79877b68fdcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c51c0a335cef4b3c8952da5cc307bbd0" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45823 } } peers { permanent_uuid: "52cb109847d5481f9c3c79877b68fdcd" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 42729 } } peers { permanent_uuid: "6756602d01cc40708be44ba8e55ed744" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37155 } }
I20260808 03:56:07.467367 22424 consensus_queue.cc:260] T 8ff6cc42c67b492d9e4e787b1eebef96 P 6756602d01cc40708be44ba8e55ed744 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c51c0a335cef4b3c8952da5cc307bbd0" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45823 } } peers { permanent_uuid: "52cb109847d5481f9c3c79877b68fdcd" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 42729 } } peers { permanent_uuid: "6756602d01cc40708be44ba8e55ed744" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37155 } }
I20260808 03:56:07.468595 22422 ts_tablet_manager.cc:1428] T 8ff6cc42c67b492d9e4e787b1eebef96 P 52cb109847d5481f9c3c79877b68fdcd: Time spent starting tablet: real 0.012s user 0.010s sys 0.001s
I20260808 03:56:07.474308 22394 heartbeater.cc:503] Master 127.26.146.190:45029 was elected leader, sending a full tablet report...
I20260808 03:56:07.474957 22424 ts_tablet_manager.cc:1428] T 8ff6cc42c67b492d9e4e787b1eebef96 P 6756602d01cc40708be44ba8e55ed744: Time spent starting tablet: real 0.017s user 0.006s sys 0.012s
W20260808 03:56:07.550627 22013 tablet.cc:2369] T 8ff6cc42c67b492d9e4e787b1eebef96 P 52cb109847d5481f9c3c79877b68fdcd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:56:07.572203 22210 tablet.cc:2369] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:56:07.579047 22427 raft_consensus.cc:482] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:56:07.579335 22427 raft_consensus.cc:504] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c51c0a335cef4b3c8952da5cc307bbd0" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45823 } } peers { permanent_uuid: "52cb109847d5481f9c3c79877b68fdcd" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 42729 } } peers { permanent_uuid: "6756602d01cc40708be44ba8e55ed744" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37155 } }
I20260808 03:56:07.581110 22427 leader_election.cc:288] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52cb109847d5481f9c3c79877b68fdcd (127.26.146.129:42729), 6756602d01cc40708be44ba8e55ed744 (127.26.146.131:37155)
W20260808 03:56:07.584388 22397 tablet.cc:2369] T 8ff6cc42c67b492d9e4e787b1eebef96 P 6756602d01cc40708be44ba8e55ed744: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:56:07.589289 22418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:07.581572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:58169 (local address 127.26.146.131:37155)
0808 03:56:07.581766 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:07.581772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:07.582517 (+ 745us) server_negotiation.cc:407] Connection header received
0808 03:56:07.582678 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:07.582681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:07.582741 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:07.582815 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:07.583413 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.584235 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.587012 (+ 2777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.588247 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.588407 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:07.588482 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:07.588922 (+ 440us) server_negotiation.cc:299] Negotiation successful
0808 03:56:07.589045 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 03:56:07.589293 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:07.581503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:52369 (local address 127.26.146.129:42729)
0808 03:56:07.581758 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:07.581764 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:07.582438 (+ 674us) server_negotiation.cc:407] Connection header received
0808 03:56:07.582620 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:07.582625 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:07.582685 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:07.582759 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:07.583392 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.584179 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.587017 (+ 2838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.588247 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.588407 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:07.588482 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:07.588922 (+ 440us) server_negotiation.cc:299] Negotiation successful
0808 03:56:07.589045 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260808 03:56:07.589536 22433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:07.581553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:37155 (local address 127.26.146.130:58169)
0808 03:56:07.582413 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:07.582427 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:07.582550 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:07.582841 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:07.582844 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:07.582871 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:07.583263 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:07.583272 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.584355 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.584363 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:07.586838 (+ 2475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:07.586858 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.588499 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.588505 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:07.588760 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:07.588805 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 03:56:07.588896 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 03:56:07.589084 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"mutex_wait_us":127,"thread_start_us":450,"threads_started":1}
I20260808 03:56:07.589641 22436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:07.581303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:42729 (local address 127.26.146.130:52369)
0808 03:56:07.582330 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:07.582345 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:07.582484 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:07.582786 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:07.582790 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:07.582814 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:07.583249 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:07.583260 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.584308 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.584313 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:07.586871 (+ 2558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:07.586880 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.588494 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.588502 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:07.588752 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:07.588798 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 03:56:07.588896 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 03:56:07.589077 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":918,"mutex_wait_us":209,"thread_start_us":743,"threads_started":1}
I20260808 03:56:07.590487 22334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ff6cc42c67b492d9e4e787b1eebef96" candidate_uuid: "c51c0a335cef4b3c8952da5cc307bbd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6756602d01cc40708be44ba8e55ed744" is_pre_election: true
I20260808 03:56:07.590510 21946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ff6cc42c67b492d9e4e787b1eebef96" candidate_uuid: "c51c0a335cef4b3c8952da5cc307bbd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52cb109847d5481f9c3c79877b68fdcd" is_pre_election: true
I20260808 03:56:07.590941 22334 raft_consensus.cc:2454] T 8ff6cc42c67b492d9e4e787b1eebef96 P 6756602d01cc40708be44ba8e55ed744 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c51c0a335cef4b3c8952da5cc307bbd0 in term 0.
I20260808 03:56:07.590970 21946 raft_consensus.cc:2454] T 8ff6cc42c67b492d9e4e787b1eebef96 P 52cb109847d5481f9c3c79877b68fdcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c51c0a335cef4b3c8952da5cc307bbd0 in term 0.
I20260808 03:56:07.591566 22046 leader_election.cc:302] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6756602d01cc40708be44ba8e55ed744, c51c0a335cef4b3c8952da5cc307bbd0; no voters:
I20260808 03:56:07.591939 22427 raft_consensus.cc:2781] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:56:07.592095 22427 raft_consensus.cc:482] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:56:07.592195 22427 raft_consensus.cc:3037] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:07.594322 22427 raft_consensus.cc:504] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c51c0a335cef4b3c8952da5cc307bbd0" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45823 } } peers { permanent_uuid: "52cb109847d5481f9c3c79877b68fdcd" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 42729 } } peers { permanent_uuid: "6756602d01cc40708be44ba8e55ed744" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37155 } }
I20260808 03:56:07.594887 22427 leader_election.cc:288] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0 [CANDIDATE]: Term 1 election: Requested vote from peers 52cb109847d5481f9c3c79877b68fdcd (127.26.146.129:42729), 6756602d01cc40708be44ba8e55ed744 (127.26.146.131:37155)
I20260808 03:56:07.595294 21946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ff6cc42c67b492d9e4e787b1eebef96" candidate_uuid: "c51c0a335cef4b3c8952da5cc307bbd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52cb109847d5481f9c3c79877b68fdcd"
I20260808 03:56:07.595405 22334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ff6cc42c67b492d9e4e787b1eebef96" candidate_uuid: "c51c0a335cef4b3c8952da5cc307bbd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6756602d01cc40708be44ba8e55ed744"
I20260808 03:56:07.595484 21946 raft_consensus.cc:3037] T 8ff6cc42c67b492d9e4e787b1eebef96 P 52cb109847d5481f9c3c79877b68fdcd [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:07.595592 22334 raft_consensus.cc:3037] T 8ff6cc42c67b492d9e4e787b1eebef96 P 6756602d01cc40708be44ba8e55ed744 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:07.597678 21946 raft_consensus.cc:2454] T 8ff6cc42c67b492d9e4e787b1eebef96 P 52cb109847d5481f9c3c79877b68fdcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c51c0a335cef4b3c8952da5cc307bbd0 in term 1.
I20260808 03:56:07.597743 22334 raft_consensus.cc:2454] T 8ff6cc42c67b492d9e4e787b1eebef96 P 6756602d01cc40708be44ba8e55ed744 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c51c0a335cef4b3c8952da5cc307bbd0 in term 1.
I20260808 03:56:07.598061 22043 leader_election.cc:302] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52cb109847d5481f9c3c79877b68fdcd, c51c0a335cef4b3c8952da5cc307bbd0; no voters:
I20260808 03:56:07.598347 22427 raft_consensus.cc:2781] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:07.598778 22427 raft_consensus.cc:686] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0 [term 1 LEADER]: Becoming Leader. State: Replica: c51c0a335cef4b3c8952da5cc307bbd0, State: Running, Role: LEADER
I20260808 03:56:07.599215 22427 consensus_queue.cc:237] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c51c0a335cef4b3c8952da5cc307bbd0" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45823 } } peers { permanent_uuid: "52cb109847d5481f9c3c79877b68fdcd" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 42729 } } peers { permanent_uuid: "6756602d01cc40708be44ba8e55ed744" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37155 } }
I20260808 03:56:07.603986 21722 catalog_manager.cc:5495] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0 reported cstate change: term changed from 0 to 1, leader changed from <none> to c51c0a335cef4b3c8952da5cc307bbd0 (127.26.146.130). New cstate: current_term: 1 leader_uuid: "c51c0a335cef4b3c8952da5cc307bbd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c51c0a335cef4b3c8952da5cc307bbd0" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52cb109847d5481f9c3c79877b68fdcd" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 42729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6756602d01cc40708be44ba8e55ed744" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37155 } health_report { overall_health: UNKNOWN } } }
I20260808 03:56:07.623145 22413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:07.617407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48096 (local address 127.26.146.130:45823)
0808 03:56:07.617617 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:07.617624 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:07.617767 (+ 143us) server_negotiation.cc:407] Connection header received
0808 03:56:07.617978 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:07.617983 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:07.618091 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:07.618221 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:56:07.618643 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.619346 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.620876 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.621294 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.621404 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:07.622054 (+ 650us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:56:07.622544 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:56:07.622704 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:07.622862 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 03:56:07.622907 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 03:56:07.631007 22334 raft_consensus.cc:1261] T 8ff6cc42c67b492d9e4e787b1eebef96 P 6756602d01cc40708be44ba8e55ed744 [term 1 FOLLOWER]: Refusing update from remote peer c51c0a335cef4b3c8952da5cc307bbd0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:56:07.631026 21946 raft_consensus.cc:1261] T 8ff6cc42c67b492d9e4e787b1eebef96 P 52cb109847d5481f9c3c79877b68fdcd [term 1 FOLLOWER]: Refusing update from remote peer c51c0a335cef4b3c8952da5cc307bbd0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:56:07.631915 22437 consensus_queue.cc:1035] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6756602d01cc40708be44ba8e55ed744" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 37155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:56:07.632256 22427 consensus_queue.cc:1035] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52cb109847d5481f9c3c79877b68fdcd" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 42729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:56:07.670552 21723 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:56:07.670831 21723 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:56:07.674293 21723 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f166c8e5dfb47c6a0525522363ba0de: Sending DeleteTablet for 3 replicas of tablet 8ff6cc42c67b492d9e4e787b1eebef96
I20260808 03:56:07.675408 22114 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ff6cc42c67b492d9e4e787b1eebef96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:56:07 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:48080
I20260808 03:56:07.675571 21917 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ff6cc42c67b492d9e4e787b1eebef96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:56:07 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:60150
I20260808 03:56:07.675827 22307 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ff6cc42c67b492d9e4e787b1eebef96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:56:07 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:55966
I20260808 03:56:07.680503 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 21806
I20260808 03:56:07.709856 22455 tablet_replica.cc:330] T 8ff6cc42c67b492d9e4e787b1eebef96 P 6756602d01cc40708be44ba8e55ed744: stopping tablet replica
I20260808 03:56:07.710491 22455 raft_consensus.cc:2229] T 8ff6cc42c67b492d9e4e787b1eebef96 P 6756602d01cc40708be44ba8e55ed744 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:56:07.711215 22455 raft_consensus.cc:2258] T 8ff6cc42c67b492d9e4e787b1eebef96 P 6756602d01cc40708be44ba8e55ed744 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:56:07.717455 22455 ts_tablet_manager.cc:1905] T 8ff6cc42c67b492d9e4e787b1eebef96 P 6756602d01cc40708be44ba8e55ed744: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:56:07.727375 22455 ts_tablet_manager.cc:1918] T 8ff6cc42c67b492d9e4e787b1eebef96 P 6756602d01cc40708be44ba8e55ed744: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 08 03:56:07 dist-test-slave-jlzn krb5kdc[21653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.129@KRBTEST.COM
I20260808 03:56:07.732100 22452 tablet_replica.cc:330] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0: stopping tablet replica
I20260808 03:56:07.732335 22453 tablet_replica.cc:330] T 8ff6cc42c67b492d9e4e787b1eebef96 P 52cb109847d5481f9c3c79877b68fdcd: stopping tablet replica
I20260808 03:56:07.732560 22452 raft_consensus.cc:2229] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0 [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:56:07.732766 22453 raft_consensus.cc:2229] T 8ff6cc42c67b492d9e4e787b1eebef96 P 52cb109847d5481f9c3c79877b68fdcd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:56:07.733237 22452 raft_consensus.cc:2258] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:56:07.733295 22453 raft_consensus.cc:2258] T 8ff6cc42c67b492d9e4e787b1eebef96 P 52cb109847d5481f9c3c79877b68fdcd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:56:07.734831 22455 log.cc:1196] T 8ff6cc42c67b492d9e4e787b1eebef96 P 6756602d01cc40708be44ba8e55ed744: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/8ff6cc42c67b492d9e4e787b1eebef96
I20260808 03:56:07.735353 22452 ts_tablet_manager.cc:1905] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:56:07.735411 22455 ts_tablet_manager.cc:1939] T 8ff6cc42c67b492d9e4e787b1eebef96 P 6756602d01cc40708be44ba8e55ed744: Deleting consensus metadata
I20260808 03:56:07.737371 21698 catalog_manager.cc:4841] TS 6756602d01cc40708be44ba8e55ed744 (127.26.146.131:37155): tablet 8ff6cc42c67b492d9e4e787b1eebef96 (table test-table [id=26f269a53fc6447dab23b68d0f087e5e]) successfully deleted
I20260808 03:56:07.740756 22453 ts_tablet_manager.cc:1905] T 8ff6cc42c67b492d9e4e787b1eebef96 P 52cb109847d5481f9c3c79877b68fdcd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:56:07.744096 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:07.681887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60152 (local address 127.26.146.129:42729)
0808 03:56:07.682063 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:07.682070 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:07.706665 (+ 24595us) server_negotiation.cc:407] Connection header received
0808 03:56:07.707197 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:07.707204 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:07.707304 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:07.707381 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:07.711468 (+ 4087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.712845 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.717825 (+ 4980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.718590 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.718696 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:07.734832 (+ 16136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:07.734847 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:07.734865 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:07.734913 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:07.738600 (+ 3687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:07.739586 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:07.739595 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:07.739598 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:07.739772 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:07.740231 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:07.740237 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:07.740241 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:07.740753 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:07.741079 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:07.743405 (+ 2326us) server_negotiation.cc:299] Negotiation successful
0808 03:56:07.743658 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 03:56:07.745527 22005 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:07.745427 22452 ts_tablet_manager.cc:1918] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:56:07.745873 22452 log.cc:1196] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/8ff6cc42c67b492d9e4e787b1eebef96
I20260808 03:56:07.746439 22452 ts_tablet_manager.cc:1939] T 8ff6cc42c67b492d9e4e787b1eebef96 P c51c0a335cef4b3c8952da5cc307bbd0: Deleting consensus metadata
I20260808 03:56:07.748591 21695 catalog_manager.cc:4841] TS c51c0a335cef4b3c8952da5cc307bbd0 (127.26.146.130:45823): tablet 8ff6cc42c67b492d9e4e787b1eebef96 (table test-table [id=26f269a53fc6447dab23b68d0f087e5e]) successfully deleted
I20260808 03:56:07.842754 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 21806
W20260808 03:56:07.863198 21696 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.146.129:42729 (error 108)
I20260808 03:56:07.863317 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 22019
I20260808 03:56:07.865504 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:07.865001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:42729 (local address 127.0.0.1:60158)
0808 03:56:07.865200 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:07.865330 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:42729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260808 03:56:07.865765 21696 catalog_manager.cc:4584] TS 52cb109847d5481f9c3c79877b68fdcd (127.26.146.129:42729): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8ff6cc42c67b492d9e4e787b1eebef96: Network error: Client connection negotiation failed: client connection to 127.26.146.129:42729: connect: Connection refused (error 111)
Aug 08 03:56:07 dist-test-slave-jlzn krb5kdc[21653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.130@KRBTEST.COM
I20260808 03:56:07.880234 22413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:07.864116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48102 (local address 127.26.146.130:45823)
0808 03:56:07.864327 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:07.864334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:07.864725 (+ 391us) server_negotiation.cc:407] Connection header received
0808 03:56:07.864959 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:07.864965 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:07.865068 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:07.865168 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:07.866697 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.867481 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.868907 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:07.869350 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:07.869460 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:07.873929 (+ 4469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:07.873940 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:07.873955 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:07.873987 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:07.877470 (+ 3483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:07.878274 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:07.878284 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:07.878286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:07.878431 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:07.878807 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:07.878811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:07.878813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:07.879177 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:07.879439 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:07.879713 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 03:56:07.879917 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 03:56:07.880947 22202 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:07.919601 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:07.919029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:42729 (local address 127.0.0.1:60162)
0808 03:56:07.919278 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:07.919395 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:42729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 03:56:07.920702 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:07.920310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:42729 (local address 127.0.0.1:60170)
0808 03:56:07.920451 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:07.920538 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:42729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 03:56:07.981238 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:07.980620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:42729 (local address 127.0.0.1:60184)
0808 03:56:07.980888 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:07.981014 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:42729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260808 03:56:07.982311 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:07.981925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:42729 (local address 127.0.0.1:60192)
0808 03:56:07.982079 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:07.982166 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:42729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 03:56:07.994540 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 22019
I20260808 03:56:08.018654 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 22215
Aug 08 03:56:08 dist-test-slave-jlzn krb5kdc[21653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.131@KRBTEST.COM
I20260808 03:56:08.032944 22418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:08.019359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55972 (local address 127.26.146.131:37155)
0808 03:56:08.019553 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:08.019558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:08.019577 (+ 19us) server_negotiation.cc:407] Connection header received
0808 03:56:08.019713 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:08.019717 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:08.019778 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:08.019889 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:08.021183 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:08.021907 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:08.023124 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:08.023515 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:08.023608 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:08.027865 (+ 4257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:08.027882 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:08.027898 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:08.027937 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:08.030576 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:08.031138 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:08.031144 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:08.031146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:08.031289 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:08.031570 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:08.031573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:08.031575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:08.032040 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:08.032270 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:08.032475 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 03:56:08.032633 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 03:56:08.033574 22383 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:08.062884 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:08.062401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:42729 (local address 127.0.0.1:60208)
0808 03:56:08.062612 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:08.062710 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:42729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 03:56:08.063952 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:08.063524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:42729 (local address 127.0.0.1:60218)
0808 03:56:08.063726 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:08.063828 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:42729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 03:56:08.121767 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 22215
I20260808 03:56:08.142621 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 21674
I20260808 03:56:08.143731 21765 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:08.252938 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:08.252323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:42729 (local address 127.0.0.1:60222)
0808 03:56:08.252516 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:08.252621 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.129:42729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 03:56:08.253515 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 21674
2026-08-08T03:56:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4528 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T03:56:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:56:08Z Disabled control of system clock
I20260808 03:56:08.339764 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:38081
--webserver_interface=127.26.146.190
--webserver_port=0
--builtin_ntp_servers=127.26.146.148:33659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:38081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 03:56:08.666214 22470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:08.666500 22470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:08.666591 22470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:08.673705 22470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:08.673808 22470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:08.673846 22470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:08.673902 22470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:08.673945 22470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:08.682885 22470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:33659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:38081
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:38081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:08.684098 22470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:08.685822 22470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:08.697003 22480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:08.697062 22476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:08.697016 22475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:08.697923 22470 server_base.cc:1034] running on GCE node
I20260808 03:56:08.698685 22470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:08.700073 22470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:08.701278 22470 hybrid_clock.cc:648] HybridClock initialized: now 1786161368701240 us; error 31 us; skew 500 ppm
I20260808 03:56:08.701602 22470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:56:08.703840 22470 webserver.cc:460] Webserver started at http://127.26.146.190:35073/ using document root <none> and password file <none>
I20260808 03:56:08.704548 22470 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:08.704656 22470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:08.704955 22470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:08.707738 22470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "278dff512e6d4e4e89b48b2b09b61673"
format_stamp: "Formatted at 2026-08-08 03:56:08 on dist-test-slave-jlzn"
server_key: "cbdd64cce86fa52cadfa43c98b2de9ac"
server_key_iv: "2c3b78ed592e20a2ccc95379b5fb6284"
server_key_version: "encryptionkey@0"
I20260808 03:56:08.708559 22470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "278dff512e6d4e4e89b48b2b09b61673"
format_stamp: "Formatted at 2026-08-08 03:56:08 on dist-test-slave-jlzn"
server_key: "cbdd64cce86fa52cadfa43c98b2de9ac"
server_key_iv: "2c3b78ed592e20a2ccc95379b5fb6284"
server_key_version: "encryptionkey@0"
I20260808 03:56:08.713459 22470 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 03:56:08.716784 22489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:08.718029 22470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 03:56:08.718195 22470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "278dff512e6d4e4e89b48b2b09b61673"
format_stamp: "Formatted at 2026-08-08 03:56:08 on dist-test-slave-jlzn"
server_key: "cbdd64cce86fa52cadfa43c98b2de9ac"
server_key_iv: "2c3b78ed592e20a2ccc95379b5fb6284"
server_key_version: "encryptionkey@0"
I20260808 03:56:08.718385 22470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:08.738802 22470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:08.739661 22470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:08.739996 22470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:08.757418 22470 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:38081
I20260808 03:56:08.757437 22558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:38081 every 8 connection(s)
I20260808 03:56:08.758740 22470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:56:08.762614 22559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:08.762676 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 22470
I20260808 03:56:08.762833 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:56:08.763257 27210 external_mini_cluster.cc:1428] Setting key e1f74ee6c2458f0687d069e3a107c386
I20260808 03:56:08.775431 22559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 278dff512e6d4e4e89b48b2b09b61673: Bootstrap starting.
I20260808 03:56:08.775964 22565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:08.765077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41688 (local address 127.26.146.190:38081)
0808 03:56:08.766085 (+ 1008us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:08.766096 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:08.766138 (+ 42us) server_negotiation.cc:407] Connection header received
0808 03:56:08.767643 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:08.767664 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:08.768140 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:08.768479 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:08.769058 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:08.770339 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:08.771679 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:08.772391 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:08.772580 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:08.773725 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:08.773733 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:08.773745 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:08.774294 (+ 549us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:08.774368 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:08.774389 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:08.774494 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:08.774716 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 03:56:08.774998 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":373,"threads_started":1}
I20260808 03:56:08.778929 22559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 278dff512e6d4e4e89b48b2b09b61673: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:08.779922 22559 log.cc:824] T 00000000000000000000000000000000 P 278dff512e6d4e4e89b48b2b09b61673: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:08.782446 22559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 278dff512e6d4e4e89b48b2b09b61673: No bootstrap required, opened a new log
I20260808 03:56:08.789847 22559 raft_consensus.cc:348] T 00000000000000000000000000000000 P 278dff512e6d4e4e89b48b2b09b61673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "278dff512e6d4e4e89b48b2b09b61673" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38081 } }
I20260808 03:56:08.790191 22559 raft_consensus.cc:374] T 00000000000000000000000000000000 P 278dff512e6d4e4e89b48b2b09b61673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:08.790280 22559 raft_consensus.cc:729] T 00000000000000000000000000000000 P 278dff512e6d4e4e89b48b2b09b61673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 278dff512e6d4e4e89b48b2b09b61673, State: Initialized, Role: FOLLOWER
I20260808 03:56:08.790822 22559 consensus_queue.cc:260] T 00000000000000000000000000000000 P 278dff512e6d4e4e89b48b2b09b61673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "278dff512e6d4e4e89b48b2b09b61673" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38081 } }
I20260808 03:56:08.791049 22559 raft_consensus.cc:388] T 00000000000000000000000000000000 P 278dff512e6d4e4e89b48b2b09b61673 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:56:08.791177 22559 raft_consensus.cc:482] T 00000000000000000000000000000000 P 278dff512e6d4e4e89b48b2b09b61673 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:56:08.791318 22559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 278dff512e6d4e4e89b48b2b09b61673 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:08.793457 22559 raft_consensus.cc:504] T 00000000000000000000000000000000 P 278dff512e6d4e4e89b48b2b09b61673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "278dff512e6d4e4e89b48b2b09b61673" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38081 } }
I20260808 03:56:08.793941 22559 leader_election.cc:302] T 00000000000000000000000000000000 P 278dff512e6d4e4e89b48b2b09b61673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 278dff512e6d4e4e89b48b2b09b61673; no voters:
I20260808 03:56:08.794498 22559 leader_election.cc:288] T 00000000000000000000000000000000 P 278dff512e6d4e4e89b48b2b09b61673 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:56:08.794624 22568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 278dff512e6d4e4e89b48b2b09b61673 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:08.795293 22568 raft_consensus.cc:686] T 00000000000000000000000000000000 P 278dff512e6d4e4e89b48b2b09b61673 [term 1 LEADER]: Becoming Leader. State: Replica: 278dff512e6d4e4e89b48b2b09b61673, State: Running, Role: LEADER
I20260808 03:56:08.795773 22568 consensus_queue.cc:237] T 00000000000000000000000000000000 P 278dff512e6d4e4e89b48b2b09b61673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "278dff512e6d4e4e89b48b2b09b61673" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38081 } }
I20260808 03:56:08.796530 22559 sys_catalog.cc:564] T 00000000000000000000000000000000 P 278dff512e6d4e4e89b48b2b09b61673 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:08.798722 22574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 278dff512e6d4e4e89b48b2b09b61673 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 278dff512e6d4e4e89b48b2b09b61673. Latest consensus state: current_term: 1 leader_uuid: "278dff512e6d4e4e89b48b2b09b61673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "278dff512e6d4e4e89b48b2b09b61673" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38081 } } }
I20260808 03:56:08.799191 22574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 278dff512e6d4e4e89b48b2b09b61673 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:08.799043 22573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 278dff512e6d4e4e89b48b2b09b61673 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "278dff512e6d4e4e89b48b2b09b61673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "278dff512e6d4e4e89b48b2b09b61673" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 38081 } } }
I20260808 03:56:08.799343 22573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 278dff512e6d4e4e89b48b2b09b61673 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:08.800566 22582 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:56:08.805583 22582 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:56:08.814333 22582 catalog_manager.cc:1347] Generated new cluster ID: 77e5a434bf084bf4adf9c8a671b8200d
I20260808 03:56:08.814455 22582 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:56:08.837862 22582 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:56:08.839706 22582 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:56:08.856459 22582 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 278dff512e6d4e4e89b48b2b09b61673: Generated new TSK 0
I20260808 03:56:08.857533 22582 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:56:08.874980 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:38081
--builtin_ntp_servers=127.26.146.148:33659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 03:56:09.213032 22594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:09.213359 22594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:09.213477 22594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:09.220783 22594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:09.220906 22594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:09.221065 22594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:56:09.230499 22594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:33659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:38081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:09.231904 22594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:09.233690 22594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:09.246464 22602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:09.246522 22601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:09.246492 22607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:09.247237 22594 server_base.cc:1034] running on GCE node
I20260808 03:56:09.247865 22594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:09.248795 22594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:09.249997 22594 hybrid_clock.cc:648] HybridClock initialized: now 1786161369249954 us; error 32 us; skew 500 ppm
I20260808 03:56:09.250298 22594 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:56:09.252521 22594 webserver.cc:460] Webserver started at http://127.26.146.129:44173/ using document root <none> and password file <none>
I20260808 03:56:09.253180 22594 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:09.253289 22594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:09.253602 22594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:09.256464 22594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "a00101bb8eec486aa2aba09ed4bfbbab"
format_stamp: "Formatted at 2026-08-08 03:56:09 on dist-test-slave-jlzn"
server_key: "bde97237f1446d059f97efab6bb131dd"
server_key_iv: "8e98064495b8324ac00ea5e0eb54641e"
server_key_version: "encryptionkey@0"
I20260808 03:56:09.257230 22594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "a00101bb8eec486aa2aba09ed4bfbbab"
format_stamp: "Formatted at 2026-08-08 03:56:09 on dist-test-slave-jlzn"
server_key: "bde97237f1446d059f97efab6bb131dd"
server_key_iv: "8e98064495b8324ac00ea5e0eb54641e"
server_key_version: "encryptionkey@0"
I20260808 03:56:09.262130 22594 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:56:09.265431 22615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:09.266672 22594 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 03:56:09.266827 22594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "a00101bb8eec486aa2aba09ed4bfbbab"
format_stamp: "Formatted at 2026-08-08 03:56:09 on dist-test-slave-jlzn"
server_key: "bde97237f1446d059f97efab6bb131dd"
server_key_iv: "8e98064495b8324ac00ea5e0eb54641e"
server_key_version: "encryptionkey@0"
I20260808 03:56:09.267021 22594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:09.297741 22594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:09.298635 22594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:09.298920 22594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:09.299985 22594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:56:09.301431 22594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:56:09.301529 22594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:09.301654 22594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:56:09.301736 22594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:09.331480 22594 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:40363
I20260808 03:56:09.331552 22781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:40363 every 8 connection(s)
I20260808 03:56:09.332814 22594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:56:09.337189 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 22594
I20260808 03:56:09.337361 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:56:09.337911 27210 external_mini_cluster.cc:1428] Setting key 97c3581ddb6e472fb5bdc581419b1bf7
I20260808 03:56:09.340893 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:38081
--builtin_ntp_servers=127.26.146.148:33659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 03:56:09.345889 22784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:09.335161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:55673 (local address 127.26.146.190:38081)
0808 03:56:09.335583 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:09.335587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:09.336694 (+ 1107us) server_negotiation.cc:407] Connection header received
0808 03:56:09.339000 (+ 2306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:09.339007 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:09.339106 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:09.339222 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:09.340457 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:09.341249 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:09.343324 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:09.343699 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:09.343788 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:09.345203 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:09.345209 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:09.345212 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:09.345230 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:09.345269 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:09.345281 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:09.345349 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:09.345555 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 03:56:09.345631 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":216,"threads_started":1}
I20260808 03:56:09.347127 22785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:09.335653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38081 (local address 127.26.146.129:55673)
0808 03:56:09.336532 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:09.336566 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:09.338701 (+ 2135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:09.339391 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:09.339398 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:09.339477 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:09.340282 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:09.340309 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:09.341393 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:09.341399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:09.343207 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:09.343221 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:09.343926 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:09.343933 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:09.344190 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:09.344880 (+ 690us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:56:09.344883 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:09.344930 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:56:09.345021 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:09.345367 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:09.345377 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:09.345384 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 03:56:09.345543 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 03:56:09.345908 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":306,"threads_started":1}
I20260808 03:56:09.348605 22782 heartbeater.cc:347] Connected to a master server at 127.26.146.190:38081
I20260808 03:56:09.348905 22782 heartbeater.cc:464] Registering TS with master...
I20260808 03:56:09.349752 22782 heartbeater.cc:511] Master 127.26.146.190:38081 requested a full tablet report, sending...
I20260808 03:56:09.351758 22510 ts_manager.cc:194] Registered new tserver with Master: a00101bb8eec486aa2aba09ed4bfbbab (127.26.146.129:40363)
I20260808 03:56:09.353227 22510 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.146.129:55673
W20260808 03:56:09.675581 22787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:09.675884 22787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:09.676008 22787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:09.683172 22787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:09.683269 22787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:09.683414 22787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:56:09.692436 22787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:33659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:38081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:09.693708 22787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:09.695469 22787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:09.708253 22792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:09.708204 22795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:09.708281 22793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:09.708796 22787 server_base.cc:1034] running on GCE node
I20260808 03:56:09.709384 22787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:09.710260 22787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:09.711457 22787 hybrid_clock.cc:648] HybridClock initialized: now 1786161369711403 us; error 47 us; skew 500 ppm
I20260808 03:56:09.711763 22787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:56:09.713937 22787 webserver.cc:460] Webserver started at http://127.26.146.130:35583/ using document root <none> and password file <none>
I20260808 03:56:09.714576 22787 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:09.714690 22787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:09.714982 22787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:09.717770 22787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "99477e3a78c14b599908117cbc4a1d65"
format_stamp: "Formatted at 2026-08-08 03:56:09 on dist-test-slave-jlzn"
server_key: "36531cdd8e58b9942b519e3dfe2228e9"
server_key_iv: "dfb740a39e82309713eb3bf3677fb22c"
server_key_version: "encryptionkey@0"
I20260808 03:56:09.718506 22787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "99477e3a78c14b599908117cbc4a1d65"
format_stamp: "Formatted at 2026-08-08 03:56:09 on dist-test-slave-jlzn"
server_key: "36531cdd8e58b9942b519e3dfe2228e9"
server_key_iv: "dfb740a39e82309713eb3bf3677fb22c"
server_key_version: "encryptionkey@0"
I20260808 03:56:09.723469 22787 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 03:56:09.726787 22802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:09.728080 22787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 03:56:09.728232 22787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "99477e3a78c14b599908117cbc4a1d65"
format_stamp: "Formatted at 2026-08-08 03:56:09 on dist-test-slave-jlzn"
server_key: "36531cdd8e58b9942b519e3dfe2228e9"
server_key_iv: "dfb740a39e82309713eb3bf3677fb22c"
server_key_version: "encryptionkey@0"
I20260808 03:56:09.728426 22787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:09.744697 22787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:09.745496 22787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:09.745790 22787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:09.746737 22787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:56:09.748207 22787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:56:09.748309 22787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:09.748416 22787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:56:09.748487 22787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:09.779850 22787 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:44763
I20260808 03:56:09.779910 22968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:44763 every 8 connection(s)
I20260808 03:56:09.781168 22787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:56:09.781636 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 22787
I20260808 03:56:09.781798 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:56:09.782243 27210 external_mini_cluster.cc:1428] Setting key 1c7936f7a47293be017bb417d40802c3
I20260808 03:56:09.785269 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:38081
--builtin_ntp_servers=127.26.146.148:33659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 03:56:09.793906 22784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:09.783890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:54817 (local address 127.26.146.190:38081)
0808 03:56:09.784074 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:09.784079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:09.785669 (+ 1590us) server_negotiation.cc:407] Connection header received
0808 03:56:09.787454 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:09.787460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:09.787535 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:09.787663 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:09.788705 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:09.789591 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:09.791199 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:09.791738 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:09.791880 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:09.793114 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:09.793120 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:09.793124 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:09.793145 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:09.793182 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:09.793196 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:09.793281 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:09.793517 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 03:56:09.793610 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 03:56:09.794902 22973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:09.784474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38081 (local address 127.26.146.130:54817)
0808 03:56:09.785495 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:09.785557 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:09.787201 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:09.787883 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:09.787891 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:09.787981 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:09.788529 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:09.788559 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:09.789745 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:09.789749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:09.791059 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:09.791072 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:09.791959 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:09.791964 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:09.792178 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:09.792810 (+ 632us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:56:09.792813 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:09.792860 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:56:09.792932 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:09.793319 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:09.793329 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:09.793336 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 03:56:09.793492 (+ 156us) client_negotiation.cc:240] Negotiation successful
0808 03:56:09.793784 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":314,"threads_started":1}
I20260808 03:56:09.796166 22969 heartbeater.cc:347] Connected to a master server at 127.26.146.190:38081
I20260808 03:56:09.796468 22969 heartbeater.cc:464] Registering TS with master...
I20260808 03:56:09.797384 22969 heartbeater.cc:511] Master 127.26.146.190:38081 requested a full tablet report, sending...
I20260808 03:56:09.798861 22510 ts_manager.cc:194] Registered new tserver with Master: 99477e3a78c14b599908117cbc4a1d65 (127.26.146.130:44763)
I20260808 03:56:09.799621 22510 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.146.130:54817
W20260808 03:56:10.118510 22974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:10.118784 22974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:10.118877 22974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:10.126057 22974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:10.126189 22974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:10.126341 22974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:56:10.135259 22974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:33659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:38081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:10.136533 22974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:10.138298 22974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:10.150579 22982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:10.150622 22985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:10.150655 22983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:10.151747 22974 server_base.cc:1034] running on GCE node
I20260808 03:56:10.152340 22974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:10.153260 22974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:10.154456 22974 hybrid_clock.cc:648] HybridClock initialized: now 1786161370154418 us; error 29 us; skew 500 ppm
I20260808 03:56:10.154754 22974 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:56:10.159899 22974 webserver.cc:460] Webserver started at http://127.26.146.131:38683/ using document root <none> and password file <none>
I20260808 03:56:10.160595 22974 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:10.160696 22974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:10.160998 22974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:10.163862 22974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "ec0eafcf7ef3413ea6d1d9cb4a4e73c7"
format_stamp: "Formatted at 2026-08-08 03:56:10 on dist-test-slave-jlzn"
server_key: "303556dee5e60e5e62fddb801d78fbf2"
server_key_iv: "e69ed42d0acf24fd36184510379b8d44"
server_key_version: "encryptionkey@0"
I20260808 03:56:10.164654 22974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "ec0eafcf7ef3413ea6d1d9cb4a4e73c7"
format_stamp: "Formatted at 2026-08-08 03:56:10 on dist-test-slave-jlzn"
server_key: "303556dee5e60e5e62fddb801d78fbf2"
server_key_iv: "e69ed42d0acf24fd36184510379b8d44"
server_key_version: "encryptionkey@0"
I20260808 03:56:10.169550 22974 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:56:10.172914 22995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:10.174162 22974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:56:10.174310 22974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "ec0eafcf7ef3413ea6d1d9cb4a4e73c7"
format_stamp: "Formatted at 2026-08-08 03:56:10 on dist-test-slave-jlzn"
server_key: "303556dee5e60e5e62fddb801d78fbf2"
server_key_iv: "e69ed42d0acf24fd36184510379b8d44"
server_key_version: "encryptionkey@0"
I20260808 03:56:10.174495 22974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:10.188136 22974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:10.188874 22974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:10.189149 22974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:10.190022 22974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:56:10.191398 22974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:56:10.191504 22974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:10.191612 22974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:56:10.191681 22974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:10.221705 22974 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:43581
I20260808 03:56:10.221807 23149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:43581 every 8 connection(s)
I20260808 03:56:10.223189 22974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:56:10.225996 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 22974
I20260808 03:56:10.226195 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:56:10.226780 27210 external_mini_cluster.cc:1428] Setting key 1a1f7cf4cfcc247448d7f1aa3752d1d8
I20260808 03:56:10.235623 22784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:10.225694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:38229 (local address 127.26.146.190:38081)
0808 03:56:10.225880 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:10.225885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:10.227447 (+ 1562us) server_negotiation.cc:407] Connection header received
0808 03:56:10.229696 (+ 2249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:10.229702 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:10.229784 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:10.229866 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:10.230920 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.231730 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.233338 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.233750 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.233847 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:10.235017 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:10.235022 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:10.235025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:10.235041 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:10.235067 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:10.235078 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:10.235156 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:10.235295 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 03:56:10.235367 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 03:56:10.236590 23152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:10.226232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:38081 (local address 127.26.146.131:38229)
0808 03:56:10.227259 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:10.227319 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:10.229418 (+ 2099us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:10.230112 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:10.230121 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:10.230204 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:10.230760 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:10.230778 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.231897 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.231901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:10.233219 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:10.233232 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.233920 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.233925 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:10.234122 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:10.234760 (+ 638us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:56:10.234763 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:10.234800 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:56:10.234872 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:10.235148 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:10.235154 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:10.235159 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 03:56:10.235282 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 03:56:10.235544 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":341,"threads_started":1}
I20260808 03:56:10.237838 23150 heartbeater.cc:347] Connected to a master server at 127.26.146.190:38081
I20260808 03:56:10.238085 23150 heartbeater.cc:464] Registering TS with master...
I20260808 03:56:10.238768 23150 heartbeater.cc:511] Master 127.26.146.190:38081 requested a full tablet report, sending...
I20260808 03:56:10.240108 22510 ts_manager.cc:194] Registered new tserver with Master: ec0eafcf7ef3413ea6d1d9cb4a4e73c7 (127.26.146.131:43581)
I20260808 03:56:10.240932 22510 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.146.131:38229
I20260808 03:56:10.241923 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:56:10.254601 22784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:10.246968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41700 (local address 127.26.146.190:38081)
0808 03:56:10.247162 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:10.247168 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:10.247604 (+ 436us) server_negotiation.cc:407] Connection header received
0808 03:56:10.247840 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:10.247845 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:10.247925 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:10.248015 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:10.248687 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.249722 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.251421 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.251949 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.252025 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:10.253797 (+ 1772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:10.253804 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:10.253809 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:10.253831 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:10.253876 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:10.253896 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:10.254032 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:10.254299 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 03:56:10.254354 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 03:56:10.260994 22509 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:56:10.270303 22509 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:56:10.306905 23170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:10.284992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:44763 (local address 127.0.0.1:49946)
0808 03:56:10.295119 (+ 10127us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:10.295155 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:10.295349 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:10.297713 (+ 2364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:10.297721 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:10.297761 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:10.298228 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:10.298242 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.300286 (+ 2044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.300292 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:10.303667 (+ 3375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:10.303685 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.305980 (+ 2295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.305986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:10.306201 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:10.306264 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 03:56:10.306393 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 03:56:10.306575 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9870,"thread_start_us":10833,"threads_started":1}
I20260808 03:56:10.306990 23172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:10.291460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49946 (local address 127.26.146.130:44763)
0808 03:56:10.294489 (+ 3029us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:10.294499 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:10.296285 (+ 1786us) server_negotiation.cc:407] Connection header received
0808 03:56:10.296412 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:10.296421 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:10.296682 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:10.296901 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:10.298403 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.300096 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.303906 (+ 3810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.305461 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.305592 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:10.305806 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:10.306472 (+ 666us) server_negotiation.cc:299] Negotiation successful
0808 03:56:10.306653 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2861,"thread_start_us":2708,"threads_started":1}
I20260808 03:56:10.310912 23167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:10.285227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:43581 (local address 127.0.0.1:34934)
0808 03:56:10.295559 (+ 10332us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:10.295578 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:10.295699 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:10.305048 (+ 9349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:10.305053 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:10.305080 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:10.305434 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:10.305447 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.306909 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.306914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:10.308876 (+ 1962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:10.308891 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.310140 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.310147 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:10.310313 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:10.310344 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 03:56:10.310429 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 03:56:10.310574 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10162,"mutex_wait_us":5,"thread_start_us":10247,"threads_started":1}
I20260808 03:56:10.310945 23169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:10.285911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34934 (local address 127.26.146.131:43581)
0808 03:56:10.304437 (+ 18526us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:10.304447 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:10.304486 (+ 39us) server_negotiation.cc:407] Connection header received
0808 03:56:10.304585 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:10.304594 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:10.304852 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:10.305056 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:10.305627 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.306752 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.309088 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.309943 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.310134 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:10.310343 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:10.310463 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 03:56:10.310608 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18333,"thread_start_us":19351,"threads_started":1}
I20260808 03:56:10.311581 22875 tablet_service.cc:1467] Processing CreateTablet for tablet 1bd2bf7baa834a299e040476fefafcdc (DEFAULT_TABLE table=test-table [id=f615306d83424f23b0404720b3f88783]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:56:10.313349 22875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bd2bf7baa834a299e040476fefafcdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:10.314507 23064 tablet_service.cc:1467] Processing CreateTablet for tablet 1bd2bf7baa834a299e040476fefafcdc (DEFAULT_TABLE table=test-table [id=f615306d83424f23b0404720b3f88783]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:56:10.315222 23168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:10.288504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:40363 (local address 127.0.0.1:55698)
0808 03:56:10.296770 (+ 8266us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:10.296786 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:10.296881 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:10.307684 (+ 10803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:10.307692 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:10.307716 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:10.308115 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:10.308124 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.309813 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.309818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:10.312421 (+ 2603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:10.312437 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.314458 (+ 2021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.314465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:10.314641 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:10.314674 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 03:56:10.314754 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 03:56:10.314876 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8148,"mutex_wait_us":58,"thread_start_us":6205,"threads_started":1}
I20260808 03:56:10.315862 23064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bd2bf7baa834a299e040476fefafcdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:10.315990 23171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:10.288618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55698 (local address 127.26.146.129:40363)
0808 03:56:10.305894 (+ 17276us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:10.305903 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:10.305930 (+ 27us) server_negotiation.cc:407] Connection header received
0808 03:56:10.307293 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:10.307300 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:10.307502 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:10.307685 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:10.308286 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.309640 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.312657 (+ 3017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.313982 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.314129 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:10.314330 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:10.315470 (+ 1140us) server_negotiation.cc:299] Negotiation successful
0808 03:56:10.315642 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17110,"thread_start_us":18398,"threads_started":1}
I20260808 03:56:10.319614 22683 tablet_service.cc:1467] Processing CreateTablet for tablet 1bd2bf7baa834a299e040476fefafcdc (DEFAULT_TABLE table=test-table [id=f615306d83424f23b0404720b3f88783]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:56:10.321012 22683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bd2bf7baa834a299e040476fefafcdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:10.328532 23173 tablet_bootstrap.cc:492] T 1bd2bf7baa834a299e040476fefafcdc P 99477e3a78c14b599908117cbc4a1d65: Bootstrap starting.
I20260808 03:56:10.330516 23174 tablet_bootstrap.cc:492] T 1bd2bf7baa834a299e040476fefafcdc P ec0eafcf7ef3413ea6d1d9cb4a4e73c7: Bootstrap starting.
I20260808 03:56:10.333443 23173 tablet_bootstrap.cc:654] T 1bd2bf7baa834a299e040476fefafcdc P 99477e3a78c14b599908117cbc4a1d65: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:10.334261 23174 tablet_bootstrap.cc:654] T 1bd2bf7baa834a299e040476fefafcdc P ec0eafcf7ef3413ea6d1d9cb4a4e73c7: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:10.334978 23173 log.cc:824] T 1bd2bf7baa834a299e040476fefafcdc P 99477e3a78c14b599908117cbc4a1d65: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:10.335446 23174 log.cc:824] T 1bd2bf7baa834a299e040476fefafcdc P ec0eafcf7ef3413ea6d1d9cb4a4e73c7: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:10.340034 23174 tablet_bootstrap.cc:492] T 1bd2bf7baa834a299e040476fefafcdc P ec0eafcf7ef3413ea6d1d9cb4a4e73c7: No bootstrap required, opened a new log
I20260808 03:56:10.340447 23174 ts_tablet_manager.cc:1397] T 1bd2bf7baa834a299e040476fefafcdc P ec0eafcf7ef3413ea6d1d9cb4a4e73c7: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260808 03:56:10.343787 23176 tablet_bootstrap.cc:492] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab: Bootstrap starting.
I20260808 03:56:10.346650 23173 tablet_bootstrap.cc:492] T 1bd2bf7baa834a299e040476fefafcdc P 99477e3a78c14b599908117cbc4a1d65: No bootstrap required, opened a new log
I20260808 03:56:10.346908 23173 ts_tablet_manager.cc:1397] T 1bd2bf7baa834a299e040476fefafcdc P 99477e3a78c14b599908117cbc4a1d65: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.009s
I20260808 03:56:10.348161 23176 tablet_bootstrap.cc:654] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:10.349311 23176 log.cc:824] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:10.350467 23174 raft_consensus.cc:348] T 1bd2bf7baa834a299e040476fefafcdc P ec0eafcf7ef3413ea6d1d9cb4a4e73c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99477e3a78c14b599908117cbc4a1d65" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 44763 } } peers { permanent_uuid: "ec0eafcf7ef3413ea6d1d9cb4a4e73c7" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 43581 } } peers { permanent_uuid: "a00101bb8eec486aa2aba09ed4bfbbab" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40363 } }
I20260808 03:56:10.350960 23174 raft_consensus.cc:374] T 1bd2bf7baa834a299e040476fefafcdc P ec0eafcf7ef3413ea6d1d9cb4a4e73c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:10.351109 23174 raft_consensus.cc:729] T 1bd2bf7baa834a299e040476fefafcdc P ec0eafcf7ef3413ea6d1d9cb4a4e73c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec0eafcf7ef3413ea6d1d9cb4a4e73c7, State: Initialized, Role: FOLLOWER
I20260808 03:56:10.351958 23174 consensus_queue.cc:260] T 1bd2bf7baa834a299e040476fefafcdc P ec0eafcf7ef3413ea6d1d9cb4a4e73c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99477e3a78c14b599908117cbc4a1d65" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 44763 } } peers { permanent_uuid: "ec0eafcf7ef3413ea6d1d9cb4a4e73c7" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 43581 } } peers { permanent_uuid: "a00101bb8eec486aa2aba09ed4bfbbab" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40363 } }
I20260808 03:56:10.352455 23176 tablet_bootstrap.cc:492] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab: No bootstrap required, opened a new log
I20260808 03:56:10.352792 23176 ts_tablet_manager.cc:1397] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260808 03:56:10.353070 23150 heartbeater.cc:503] Master 127.26.146.190:38081 was elected leader, sending a full tablet report...
I20260808 03:56:10.355010 23174 ts_tablet_manager.cc:1428] T 1bd2bf7baa834a299e040476fefafcdc P ec0eafcf7ef3413ea6d1d9cb4a4e73c7: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260808 03:56:10.354967 23173 raft_consensus.cc:348] T 1bd2bf7baa834a299e040476fefafcdc P 99477e3a78c14b599908117cbc4a1d65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99477e3a78c14b599908117cbc4a1d65" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 44763 } } peers { permanent_uuid: "ec0eafcf7ef3413ea6d1d9cb4a4e73c7" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 43581 } } peers { permanent_uuid: "a00101bb8eec486aa2aba09ed4bfbbab" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40363 } }
I20260808 03:56:10.355402 23173 raft_consensus.cc:374] T 1bd2bf7baa834a299e040476fefafcdc P 99477e3a78c14b599908117cbc4a1d65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:10.355536 23173 raft_consensus.cc:729] T 1bd2bf7baa834a299e040476fefafcdc P 99477e3a78c14b599908117cbc4a1d65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99477e3a78c14b599908117cbc4a1d65, State: Initialized, Role: FOLLOWER
I20260808 03:56:10.356143 22782 heartbeater.cc:503] Master 127.26.146.190:38081 was elected leader, sending a full tablet report...
I20260808 03:56:10.356153 23173 consensus_queue.cc:260] T 1bd2bf7baa834a299e040476fefafcdc P 99477e3a78c14b599908117cbc4a1d65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99477e3a78c14b599908117cbc4a1d65" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 44763 } } peers { permanent_uuid: "ec0eafcf7ef3413ea6d1d9cb4a4e73c7" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 43581 } } peers { permanent_uuid: "a00101bb8eec486aa2aba09ed4bfbbab" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40363 } }
I20260808 03:56:10.359371 22969 heartbeater.cc:503] Master 127.26.146.190:38081 was elected leader, sending a full tablet report...
I20260808 03:56:10.360102 23173 ts_tablet_manager.cc:1428] T 1bd2bf7baa834a299e040476fefafcdc P 99477e3a78c14b599908117cbc4a1d65: Time spent starting tablet: real 0.013s user 0.010s sys 0.005s
I20260808 03:56:10.362026 23176 raft_consensus.cc:348] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99477e3a78c14b599908117cbc4a1d65" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 44763 } } peers { permanent_uuid: "ec0eafcf7ef3413ea6d1d9cb4a4e73c7" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 43581 } } peers { permanent_uuid: "a00101bb8eec486aa2aba09ed4bfbbab" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40363 } }
I20260808 03:56:10.362413 23176 raft_consensus.cc:374] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:10.362517 23176 raft_consensus.cc:729] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a00101bb8eec486aa2aba09ed4bfbbab, State: Initialized, Role: FOLLOWER
I20260808 03:56:10.362990 23176 consensus_queue.cc:260] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99477e3a78c14b599908117cbc4a1d65" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 44763 } } peers { permanent_uuid: "ec0eafcf7ef3413ea6d1d9cb4a4e73c7" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 43581 } } peers { permanent_uuid: "a00101bb8eec486aa2aba09ed4bfbbab" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40363 } }
I20260808 03:56:10.364442 23176 ts_tablet_manager.cc:1428] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab: Time spent starting tablet: real 0.011s user 0.010s sys 0.001s
W20260808 03:56:10.474882 23151 tablet.cc:2369] T 1bd2bf7baa834a299e040476fefafcdc P ec0eafcf7ef3413ea6d1d9cb4a4e73c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:56:10.536434 22972 tablet.cc:2369] T 1bd2bf7baa834a299e040476fefafcdc P 99477e3a78c14b599908117cbc4a1d65: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:56:10.578969 23184 raft_consensus.cc:482] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:56:10.579309 23184 raft_consensus.cc:504] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99477e3a78c14b599908117cbc4a1d65" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 44763 } } peers { permanent_uuid: "ec0eafcf7ef3413ea6d1d9cb4a4e73c7" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 43581 } } peers { permanent_uuid: "a00101bb8eec486aa2aba09ed4bfbbab" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40363 } }
I20260808 03:56:10.581195 23184 leader_election.cc:288] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99477e3a78c14b599908117cbc4a1d65 (127.26.146.130:44763), ec0eafcf7ef3413ea6d1d9cb4a4e73c7 (127.26.146.131:43581)
W20260808 03:56:10.585230 22783 tablet.cc:2369] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:56:10.588169 23169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:10.581729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:49603 (local address 127.26.146.131:43581)
0808 03:56:10.581929 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:10.581957 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:10.582839 (+ 882us) server_negotiation.cc:407] Connection header received
0808 03:56:10.582981 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:10.582986 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:10.583060 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:10.583146 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:10.583708 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.584535 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.586439 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.587231 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.587366 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:10.587440 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:10.587774 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 03:56:10.587921 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 03:56:10.588196 23172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:10.581723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:52081 (local address 127.26.146.130:44763)
0808 03:56:10.581927 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:10.581934 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:10.582172 (+ 238us) server_negotiation.cc:407] Connection header received
0808 03:56:10.582428 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:10.582434 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:10.582519 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:10.582619 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:10.583189 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.584146 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.586515 (+ 2369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.587255 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.587406 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:10.587473 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:10.587844 (+ 371us) server_negotiation.cc:299] Negotiation successful
0808 03:56:10.587972 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 03:56:10.588297 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:10.581414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:44763 (local address 127.26.146.129:52081)
0808 03:56:10.582068 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:10.582087 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:10.582249 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:10.582652 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:10.582655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:10.582679 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:10.583021 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:10.583034 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.584292 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.584298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:10.586324 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:10.586336 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.587484 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.587491 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:10.587663 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:10.587686 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 03:56:10.587773 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 03:56:10.587920 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"mutex_wait_us":20,"thread_start_us":328,"threads_started":1}
I20260808 03:56:10.588343 23186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:10.582068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:43581 (local address 127.26.146.129:49603)
0808 03:56:10.582718 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:10.582741 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:10.582862 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:10.583189 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:10.583193 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:10.583217 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:10.583555 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:10.583568 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.584685 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.584689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:10.586250 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:10.586263 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.587428 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.587434 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:10.587609 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:10.587654 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 03:56:10.587723 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 03:56:10.587862 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"mutex_wait_us":23,"thread_start_us":316,"threads_started":1}
I20260808 03:56:10.589432 22905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bd2bf7baa834a299e040476fefafcdc" candidate_uuid: "a00101bb8eec486aa2aba09ed4bfbbab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99477e3a78c14b599908117cbc4a1d65" is_pre_election: true
I20260808 03:56:10.589493 23091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bd2bf7baa834a299e040476fefafcdc" candidate_uuid: "a00101bb8eec486aa2aba09ed4bfbbab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec0eafcf7ef3413ea6d1d9cb4a4e73c7" is_pre_election: true
I20260808 03:56:10.589931 22905 raft_consensus.cc:2454] T 1bd2bf7baa834a299e040476fefafcdc P 99477e3a78c14b599908117cbc4a1d65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a00101bb8eec486aa2aba09ed4bfbbab in term 0.
I20260808 03:56:10.589931 23091 raft_consensus.cc:2454] T 1bd2bf7baa834a299e040476fefafcdc P ec0eafcf7ef3413ea6d1d9cb4a4e73c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a00101bb8eec486aa2aba09ed4bfbbab in term 0.
I20260808 03:56:10.590534 22617 leader_election.cc:302] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99477e3a78c14b599908117cbc4a1d65, a00101bb8eec486aa2aba09ed4bfbbab; no voters:
I20260808 03:56:10.590922 23184 raft_consensus.cc:2781] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:56:10.591090 23184 raft_consensus.cc:482] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:56:10.591193 23184 raft_consensus.cc:3037] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:10.593519 23184 raft_consensus.cc:504] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99477e3a78c14b599908117cbc4a1d65" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 44763 } } peers { permanent_uuid: "ec0eafcf7ef3413ea6d1d9cb4a4e73c7" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 43581 } } peers { permanent_uuid: "a00101bb8eec486aa2aba09ed4bfbbab" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40363 } }
I20260808 03:56:10.594208 23184 leader_election.cc:288] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab [CANDIDATE]: Term 1 election: Requested vote from peers 99477e3a78c14b599908117cbc4a1d65 (127.26.146.130:44763), ec0eafcf7ef3413ea6d1d9cb4a4e73c7 (127.26.146.131:43581)
I20260808 03:56:10.594633 22905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bd2bf7baa834a299e040476fefafcdc" candidate_uuid: "a00101bb8eec486aa2aba09ed4bfbbab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99477e3a78c14b599908117cbc4a1d65"
I20260808 03:56:10.594636 23091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bd2bf7baa834a299e040476fefafcdc" candidate_uuid: "a00101bb8eec486aa2aba09ed4bfbbab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec0eafcf7ef3413ea6d1d9cb4a4e73c7"
I20260808 03:56:10.594834 23091 raft_consensus.cc:3037] T 1bd2bf7baa834a299e040476fefafcdc P ec0eafcf7ef3413ea6d1d9cb4a4e73c7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:10.594834 22905 raft_consensus.cc:3037] T 1bd2bf7baa834a299e040476fefafcdc P 99477e3a78c14b599908117cbc4a1d65 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:10.597136 23091 raft_consensus.cc:2454] T 1bd2bf7baa834a299e040476fefafcdc P ec0eafcf7ef3413ea6d1d9cb4a4e73c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a00101bb8eec486aa2aba09ed4bfbbab in term 1.
I20260808 03:56:10.597169 22905 raft_consensus.cc:2454] T 1bd2bf7baa834a299e040476fefafcdc P 99477e3a78c14b599908117cbc4a1d65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a00101bb8eec486aa2aba09ed4bfbbab in term 1.
I20260808 03:56:10.597517 22617 leader_election.cc:302] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99477e3a78c14b599908117cbc4a1d65, a00101bb8eec486aa2aba09ed4bfbbab; no voters:
I20260808 03:56:10.597795 23184 raft_consensus.cc:2781] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:10.598199 23184 raft_consensus.cc:686] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab [term 1 LEADER]: Becoming Leader. State: Replica: a00101bb8eec486aa2aba09ed4bfbbab, State: Running, Role: LEADER
I20260808 03:56:10.598676 23184 consensus_queue.cc:237] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99477e3a78c14b599908117cbc4a1d65" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 44763 } } peers { permanent_uuid: "ec0eafcf7ef3413ea6d1d9cb4a4e73c7" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 43581 } } peers { permanent_uuid: "a00101bb8eec486aa2aba09ed4bfbbab" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40363 } }
I20260808 03:56:10.602972 22510 catalog_manager.cc:5495] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab reported cstate change: term changed from 0 to 1, leader changed from <none> to a00101bb8eec486aa2aba09ed4bfbbab (127.26.146.129). New cstate: current_term: 1 leader_uuid: "a00101bb8eec486aa2aba09ed4bfbbab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99477e3a78c14b599908117cbc4a1d65" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 44763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec0eafcf7ef3413ea6d1d9cb4a4e73c7" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 43581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a00101bb8eec486aa2aba09ed4bfbbab" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 40363 } health_report { overall_health: HEALTHY } } }
I20260808 03:56:10.678423 23171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:10.672725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55712 (local address 127.26.146.129:40363)
0808 03:56:10.672953 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:10.672960 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:10.673047 (+ 87us) server_negotiation.cc:407] Connection header received
0808 03:56:10.673285 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:10.673290 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:10.673384 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:10.673515 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:56:10.673930 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.674631 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.676062 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.676481 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.676602 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:10.677169 (+ 567us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:56:10.677826 (+ 657us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:56:10.677994 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:10.678140 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 03:56:10.678185 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 03:56:10.686981 22905 raft_consensus.cc:1261] T 1bd2bf7baa834a299e040476fefafcdc P 99477e3a78c14b599908117cbc4a1d65 [term 1 FOLLOWER]: Refusing update from remote peer a00101bb8eec486aa2aba09ed4bfbbab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:56:10.686981 23091 raft_consensus.cc:1261] T 1bd2bf7baa834a299e040476fefafcdc P ec0eafcf7ef3413ea6d1d9cb4a4e73c7 [term 1 FOLLOWER]: Refusing update from remote peer a00101bb8eec486aa2aba09ed4bfbbab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:56:10.688036 23189 consensus_queue.cc:1035] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec0eafcf7ef3413ea6d1d9cb4a4e73c7" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 43581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:56:10.688421 23184 consensus_queue.cc:1035] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab [LEADER]: Connected to new peer: Peer: permanent_uuid: "99477e3a78c14b599908117cbc4a1d65" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 44763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:56:10.712566 22509 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:56:10.712810 22509 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:56:10.716425 22509 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 278dff512e6d4e4e89b48b2b09b61673: Sending DeleteTablet for 3 replicas of tablet 1bd2bf7baa834a299e040476fefafcdc
I20260808 03:56:10.717510 22683 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bd2bf7baa834a299e040476fefafcdc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:56:10 UTC) from {username='slave'} at 127.0.0.1:55698
I20260808 03:56:10.717478 22875 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bd2bf7baa834a299e040476fefafcdc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:56:10 UTC) from {username='slave'} at 127.0.0.1:49946
I20260808 03:56:10.717515 23064 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bd2bf7baa834a299e040476fefafcdc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:56:10 UTC) from {username='slave'} at 127.0.0.1:34934
I20260808 03:56:10.721930 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 22594
I20260808 03:56:10.731248 23207 tablet_replica.cc:330] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab: stopping tablet replica
I20260808 03:56:10.731940 23207 raft_consensus.cc:2229] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:56:10.732867 23207 raft_consensus.cc:2258] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:56:10.735677 23207 ts_tablet_manager.cc:1905] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:56:10.737751 23206 tablet_replica.cc:330] T 1bd2bf7baa834a299e040476fefafcdc P 99477e3a78c14b599908117cbc4a1d65: stopping tablet replica
I20260808 03:56:10.738243 23206 raft_consensus.cc:2229] T 1bd2bf7baa834a299e040476fefafcdc P 99477e3a78c14b599908117cbc4a1d65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:56:10.738840 23206 raft_consensus.cc:2258] T 1bd2bf7baa834a299e040476fefafcdc P 99477e3a78c14b599908117cbc4a1d65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:56:10.741113 23206 ts_tablet_manager.cc:1905] T 1bd2bf7baa834a299e040476fefafcdc P 99477e3a78c14b599908117cbc4a1d65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:56:10.745970 23207 ts_tablet_manager.cc:1918] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:56:10.746158 23207 log.cc:1196] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-0/wal/wals/1bd2bf7baa834a299e040476fefafcdc
I20260808 03:56:10.746709 23207 ts_tablet_manager.cc:1939] T 1bd2bf7baa834a299e040476fefafcdc P a00101bb8eec486aa2aba09ed4bfbbab: Deleting consensus metadata
I20260808 03:56:10.746768 23208 tablet_replica.cc:330] T 1bd2bf7baa834a299e040476fefafcdc P ec0eafcf7ef3413ea6d1d9cb4a4e73c7: stopping tablet replica
I20260808 03:56:10.747215 23208 raft_consensus.cc:2229] T 1bd2bf7baa834a299e040476fefafcdc P ec0eafcf7ef3413ea6d1d9cb4a4e73c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:56:10.747753 23208 raft_consensus.cc:2258] T 1bd2bf7baa834a299e040476fefafcdc P ec0eafcf7ef3413ea6d1d9cb4a4e73c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:56:10.749105 23206 ts_tablet_manager.cc:1918] T 1bd2bf7baa834a299e040476fefafcdc P 99477e3a78c14b599908117cbc4a1d65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:56:10.749349 23206 log.cc:1196] T 1bd2bf7baa834a299e040476fefafcdc P 99477e3a78c14b599908117cbc4a1d65: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/1bd2bf7baa834a299e040476fefafcdc
I20260808 03:56:10.749832 23171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:10.723637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55724 (local address 127.26.146.129:40363)
0808 03:56:10.723847 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:10.723855 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:10.740588 (+ 16733us) server_negotiation.cc:407] Connection header received
0808 03:56:10.741124 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:10.741134 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:10.741255 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:10.741342 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:10.742065 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.743401 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.745615 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.746233 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.746339 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:10.747848 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:10.747862 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:10.747881 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:10.748685 (+ 804us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:10.748745 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:10.748770 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:10.748848 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:10.749335 (+ 487us) server_negotiation.cc:299] Negotiation successful
0808 03:56:10.749449 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 03:56:10.749840 22494 catalog_manager.cc:4841] TS a00101bb8eec486aa2aba09ed4bfbbab (127.26.146.129:40363): tablet 1bd2bf7baa834a299e040476fefafcdc (table test-table [id=f615306d83424f23b0404720b3f88783]) successfully deleted
I20260808 03:56:10.750409 23206 ts_tablet_manager.cc:1939] T 1bd2bf7baa834a299e040476fefafcdc P 99477e3a78c14b599908117cbc4a1d65: Deleting consensus metadata
I20260808 03:56:10.751057 22773 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:10.751935 23208 ts_tablet_manager.cc:1905] T 1bd2bf7baa834a299e040476fefafcdc P ec0eafcf7ef3413ea6d1d9cb4a4e73c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:56:10.752353 22492 catalog_manager.cc:4841] TS 99477e3a78c14b599908117cbc4a1d65 (127.26.146.130:44763): tablet 1bd2bf7baa834a299e040476fefafcdc (table test-table [id=f615306d83424f23b0404720b3f88783]) successfully deleted
I20260808 03:56:10.758468 23208 ts_tablet_manager.cc:1918] T 1bd2bf7baa834a299e040476fefafcdc P ec0eafcf7ef3413ea6d1d9cb4a4e73c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:56:10.758620 23208 log.cc:1196] T 1bd2bf7baa834a299e040476fefafcdc P ec0eafcf7ef3413ea6d1d9cb4a4e73c7: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/1bd2bf7baa834a299e040476fefafcdc
I20260808 03:56:10.759081 23208 ts_tablet_manager.cc:1939] T 1bd2bf7baa834a299e040476fefafcdc P ec0eafcf7ef3413ea6d1d9cb4a4e73c7: Deleting consensus metadata
I20260808 03:56:10.760726 22493 catalog_manager.cc:4841] TS ec0eafcf7ef3413ea6d1d9cb4a4e73c7 (127.26.146.131:43581): tablet 1bd2bf7baa834a299e040476fefafcdc (table test-table [id=f615306d83424f23b0404720b3f88783]) successfully deleted
I20260808 03:56:10.849362 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 22594
I20260808 03:56:10.874019 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 22787
I20260808 03:56:10.880959 23172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:10.874835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49956 (local address 127.26.146.130:44763)
0808 03:56:10.874987 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:10.874992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:10.875010 (+ 18us) server_negotiation.cc:407] Connection header received
0808 03:56:10.875195 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:10.875199 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:10.875281 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:10.875366 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:10.875728 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.876504 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.877872 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.878386 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.878491 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:10.879533 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:10.879543 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:10.879559 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:10.880297 (+ 738us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:10.880352 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:10.880373 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:10.880461 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:10.880584 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 03:56:10.880687 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 03:56:10.881671 22960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:10.967959 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 22787
I20260808 03:56:10.991940 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 22974
I20260808 03:56:10.998741 23169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:10.992683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34938 (local address 127.26.146.131:43581)
0808 03:56:10.992855 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:10.992861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:10.992899 (+ 38us) server_negotiation.cc:407] Connection header received
0808 03:56:10.993101 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:10.993105 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:10.993225 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:10.993311 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:10.993745 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.994457 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.995732 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:10.996226 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:10.996338 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:10.997381 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:10.997392 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:10.997409 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:10.998019 (+ 610us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:10.998068 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:10.998088 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:10.998203 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:10.998312 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 03:56:10.998423 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 03:56:10.999351 23144 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:11.086912 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 22974
I20260808 03:56:11.107398 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 22470
I20260808 03:56:11.108528 22554 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:11.215662 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 22470
2026-08-08T03:56:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2965 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:56:11 dist-test-slave-jlzn krb5kdc[23220](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:56:11 dist-test-slave-jlzn krb5kdc[23220](info): set up 2 sockets
Aug 08 03:56:11 dist-test-slave-jlzn krb5kdc[23220](info): commencing operation
krb5kdc: starting...
W20260808 03:56:12.422447 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:56:12 dist-test-slave-jlzn krb5kdc[23220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:56:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:56:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:56:12.726361 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:45375
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:36991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:45375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:13.071075 23235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:13.071357 23235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:13.071511 23235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:13.078585 23235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:13.078691 23235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:13.078768 23235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:13.078840 23235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:13.078898 23235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:13.088805 23235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:36991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:45375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:45375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:13.090078 23235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:13.091434 23235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:13.102799 23245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:13.102770 23246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:13.102770 23251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:13.103673 23235 server_base.cc:1034] running on GCE node
I20260808 03:56:13.104491 23235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:13.105871 23235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:13.107095 23235 hybrid_clock.cc:648] HybridClock initialized: now 1786161373107037 us; error 29 us; skew 500 ppm
I20260808 03:56:13.107431 23235 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:13 dist-test-slave-jlzn krb5kdc[23220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161373, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:13.114327 23235 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:56:13.115821 23235 webserver.cc:460] Webserver started at http://127.26.146.190:41581/ using document root <none> and password file <none>
I20260808 03:56:13.116494 23235 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:13.116608 23235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:13.116863 23235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:13.119925 23235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "3ee3ab494fb34e4da419f000e3ebb2d0"
format_stamp: "Formatted at 2026-08-08 03:56:13 on dist-test-slave-jlzn"
server_key: "e17125f69b59bb8d2aef6277f8184980"
server_key_iv: "6ce53721f45a4314394ffc6a73fca97e"
server_key_version: "encryptionkey@0"
I20260808 03:56:13.120656 23235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "3ee3ab494fb34e4da419f000e3ebb2d0"
format_stamp: "Formatted at 2026-08-08 03:56:13 on dist-test-slave-jlzn"
server_key: "e17125f69b59bb8d2aef6277f8184980"
server_key_iv: "6ce53721f45a4314394ffc6a73fca97e"
server_key_version: "encryptionkey@0"
I20260808 03:56:13.125686 23235 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 03:56:13.128999 23263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:13.130367 23235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260808 03:56:13.130611 23235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "3ee3ab494fb34e4da419f000e3ebb2d0"
format_stamp: "Formatted at 2026-08-08 03:56:13 on dist-test-slave-jlzn"
server_key: "e17125f69b59bb8d2aef6277f8184980"
server_key_iv: "6ce53721f45a4314394ffc6a73fca97e"
server_key_version: "encryptionkey@0"
I20260808 03:56:13.130797 23235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:13.157197 23235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:13.161307 23235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:13.161600 23235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:13.179205 23235 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:45375
I20260808 03:56:13.179219 23343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:45375 every 8 connection(s)
I20260808 03:56:13.180614 23235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:56:13.184383 23344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:13.187278 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 23235
I20260808 03:56:13.187475 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:56:13.188060 27210 external_mini_cluster.cc:1428] Setting key cb5b0fdcb17391a700c5485dd23263aa
I20260808 03:56:13.196589 23344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ee3ab494fb34e4da419f000e3ebb2d0: Bootstrap starting.
I20260808 03:56:13.200276 23344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ee3ab494fb34e4da419f000e3ebb2d0: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:13.201587 23344 log.cc:824] T 00000000000000000000000000000000 P 3ee3ab494fb34e4da419f000e3ebb2d0: Log is configured to *not* fsync() on all Append() calls
Aug 08 03:56:13 dist-test-slave-jlzn krb5kdc[23220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:56:13.204912 23344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ee3ab494fb34e4da419f000e3ebb2d0: No bootstrap required, opened a new log
I20260808 03:56:13.212533 23347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:13.190043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55762 (local address 127.26.146.190:45375)
0808 03:56:13.190803 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:13.190814 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:13.190855 (+ 41us) server_negotiation.cc:407] Connection header received
0808 03:56:13.192578 (+ 1723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:13.192601 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:13.192997 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:13.193326 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:13.195170 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:13.196607 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:13.198458 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:13.199156 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:13.199287 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:13.204512 (+ 5225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:13.204520 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:13.204536 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:13.204614 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:13.208172 (+ 3558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:13.209169 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:13.209175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:13.209181 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:13.209351 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:13.209763 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:13.209767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:13.209769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:13.210336 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:13.210558 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:13.211017 (+ 459us) server_negotiation.cc:299] Negotiation successful
0808 03:56:13.211374 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":332,"threads_started":1}
I20260808 03:56:13.214241 23344 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ee3ab494fb34e4da419f000e3ebb2d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee3ab494fb34e4da419f000e3ebb2d0" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 45375 } }
I20260808 03:56:13.214715 23344 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ee3ab494fb34e4da419f000e3ebb2d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:13.214828 23344 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ee3ab494fb34e4da419f000e3ebb2d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ee3ab494fb34e4da419f000e3ebb2d0, State: Initialized, Role: FOLLOWER
I20260808 03:56:13.215425 23344 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ee3ab494fb34e4da419f000e3ebb2d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee3ab494fb34e4da419f000e3ebb2d0" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 45375 } }
I20260808 03:56:13.215667 23344 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ee3ab494fb34e4da419f000e3ebb2d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:56:13.215791 23344 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ee3ab494fb34e4da419f000e3ebb2d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:56:13.215965 23344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ee3ab494fb34e4da419f000e3ebb2d0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:13.218047 23344 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ee3ab494fb34e4da419f000e3ebb2d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee3ab494fb34e4da419f000e3ebb2d0" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 45375 } }
I20260808 03:56:13.218497 23344 leader_election.cc:302] T 00000000000000000000000000000000 P 3ee3ab494fb34e4da419f000e3ebb2d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ee3ab494fb34e4da419f000e3ebb2d0; no voters:
I20260808 03:56:13.219029 23344 leader_election.cc:288] T 00000000000000000000000000000000 P 3ee3ab494fb34e4da419f000e3ebb2d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:56:13.219164 23349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ee3ab494fb34e4da419f000e3ebb2d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:13.219792 23349 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ee3ab494fb34e4da419f000e3ebb2d0 [term 1 LEADER]: Becoming Leader. State: Replica: 3ee3ab494fb34e4da419f000e3ebb2d0, State: Running, Role: LEADER
I20260808 03:56:13.220463 23349 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ee3ab494fb34e4da419f000e3ebb2d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee3ab494fb34e4da419f000e3ebb2d0" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 45375 } }
I20260808 03:56:13.221127 23344 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ee3ab494fb34e4da419f000e3ebb2d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:13.223227 23350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ee3ab494fb34e4da419f000e3ebb2d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ee3ab494fb34e4da419f000e3ebb2d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee3ab494fb34e4da419f000e3ebb2d0" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 45375 } } }
I20260808 03:56:13.223201 23352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ee3ab494fb34e4da419f000e3ebb2d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ee3ab494fb34e4da419f000e3ebb2d0. Latest consensus state: current_term: 1 leader_uuid: "3ee3ab494fb34e4da419f000e3ebb2d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee3ab494fb34e4da419f000e3ebb2d0" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 45375 } } }
I20260808 03:56:13.223608 23352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ee3ab494fb34e4da419f000e3ebb2d0 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:13.223608 23350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ee3ab494fb34e4da419f000e3ebb2d0 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:13.228411 23362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:56:13.233556 23362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:56:13.242089 23362 catalog_manager.cc:1347] Generated new cluster ID: 4aa98a434e6f4e89a859ff9e957b6fe1
I20260808 03:56:13.242205 23362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:56:13.253798 23362 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:56:13.255043 23362 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:56:13.265733 23362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ee3ab494fb34e4da419f000e3ebb2d0: Generated new TSK 0
I20260808 03:56:13.266501 23362 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
Entry for principal kudu/127.26.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab.
I20260808 03:56:13.341041 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:45375
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:36991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:13.677146 23383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:13.677448 23383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:13.677597 23383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:13.684736 23383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:13.684856 23383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:13.685011 23383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:56:13.694784 23383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:36991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.129.keytab
--principal=kudu/127.26.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:45375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:13.696171 23383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:13.697527 23383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:13.710578 23393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:13.710671 23388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:13.710680 23390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:13.711337 23383 server_base.cc:1034] running on GCE node
I20260808 03:56:13.712105 23383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:13.713296 23383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:13.714586 23383 hybrid_clock.cc:648] HybridClock initialized: now 1786161373714534 us; error 42 us; skew 500 ppm
I20260808 03:56:13.714913 23383 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:13 dist-test-slave-jlzn krb5kdc[23220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161373, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:13.718194 23383 init.cc:377] Logged in from keytab as kudu/127.26.146.129@KRBTEST.COM (short username kudu)
I20260808 03:56:13.719676 23383 webserver.cc:460] Webserver started at http://127.26.146.129:38339/ using document root <none> and password file <none>
I20260808 03:56:13.720417 23383 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:13.720548 23383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:13.720853 23383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:13.723956 23383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "b84eceec70ae4e28aec897a5ee80dcc4"
format_stamp: "Formatted at 2026-08-08 03:56:13 on dist-test-slave-jlzn"
server_key: "47d444bdd689efb733c24b55279e03d7"
server_key_iv: "44322f665323fb52b109f7ddcdf63424"
server_key_version: "encryptionkey@0"
I20260808 03:56:13.724745 23383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "b84eceec70ae4e28aec897a5ee80dcc4"
format_stamp: "Formatted at 2026-08-08 03:56:13 on dist-test-slave-jlzn"
server_key: "47d444bdd689efb733c24b55279e03d7"
server_key_iv: "44322f665323fb52b109f7ddcdf63424"
server_key_version: "encryptionkey@0"
I20260808 03:56:13.730139 23383 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260808 03:56:13.733831 23401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:13.735374 23383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:56:13.735564 23383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "b84eceec70ae4e28aec897a5ee80dcc4"
format_stamp: "Formatted at 2026-08-08 03:56:13 on dist-test-slave-jlzn"
server_key: "47d444bdd689efb733c24b55279e03d7"
server_key_iv: "44322f665323fb52b109f7ddcdf63424"
server_key_version: "encryptionkey@0"
I20260808 03:56:13.735850 23383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:13.769222 23383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:13.773528 23383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:13.773783 23383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:13.774677 23383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:56:13.776105 23383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:56:13.776191 23383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:13.776324 23383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:56:13.776377 23383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:13.807890 23383 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:45813
I20260808 03:56:13.807936 23558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:45813 every 8 connection(s)
I20260808 03:56:13.809203 23383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:56:13.812109 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 23383
I20260808 03:56:13.812275 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:56:13.812765 27210 external_mini_cluster.cc:1428] Setting key 6dfe6e97fca3c59d19e8617f0db429fd
Aug 08 03:56:13 dist-test-slave-jlzn krb5kdc[23220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161373, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.129@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:56:13.830804 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:13.811654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:51405 (local address 127.26.146.190:45375)
0808 03:56:13.812157 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:13.812163 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:13.813328 (+ 1165us) server_negotiation.cc:407] Connection header received
0808 03:56:13.815307 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:13.815313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:13.815394 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:13.815510 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:13.817279 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:13.818251 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:13.819928 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:13.820461 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:13.820587 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:13.824481 (+ 3894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:13.824487 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:13.824492 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:13.824531 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:13.827421 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:13.828263 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:13.828268 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:13.828271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:13.828423 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:13.828865 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:13.828869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:13.828872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:13.829259 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:13.829393 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:13.830298 (+ 905us) server_negotiation.cc:299] Negotiation successful
0808 03:56:13.830479 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":259,"threads_started":1}
I20260808 03:56:13.832188 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:13.812290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:45375 (local address 127.26.146.129:51405)
0808 03:56:13.813140 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:13.813196 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:13.815038 (+ 1842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:13.815889 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:13.815898 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:13.816464 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:13.817100 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:13.817119 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:13.818435 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:13.818439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:13.819742 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:13.819755 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:13.820761 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:13.820768 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:13.820985 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:13.821716 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:56:13.821746 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:13.824246 (+ 2500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:13.827569 (+ 3323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:13.827576 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:13.827594 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:13.828135 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:13.828515 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:13.828518 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:13.828520 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:13.828748 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:13.829413 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:13.829421 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:13.829865 (+ 444us) client_negotiation.cc:769] Sending connection context
0808 03:56:13.830167 (+ 302us) client_negotiation.cc:240] Negotiation successful
0808 03:56:13.830694 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":297,"threads_started":1}
I20260808 03:56:13.833752 23561 heartbeater.cc:347] Connected to a master server at 127.26.146.190:45375
I20260808 03:56:13.834092 23561 heartbeater.cc:464] Registering TS with master...
I20260808 03:56:13.834978 23561 heartbeater.cc:511] Master 127.26.146.190:45375 requested a full tablet report, sending...
I20260808 03:56:13.837157 23291 ts_manager.cc:194] Registered new tserver with Master: b84eceec70ae4e28aec897a5ee80dcc4 (127.26.146.129:45813)
I20260808 03:56:13.838619 23291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.129@KRBTEST.COM'} at 127.26.146.129:51405
WARNING: no policy specified for kudu/127.26.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
Entry for principal kudu/127.26.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab.
I20260808 03:56:13.881251 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:45375
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:36991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:14.221073 23568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:14.221354 23568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:14.221489 23568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:14.228790 23568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:14.228906 23568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:14.229090 23568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:56:14.239148 23568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:36991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.130.keytab
--principal=kudu/127.26.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:45375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:14.240548 23568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:14.241896 23568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:14.256425 23578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:14.256676 23581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:14.256394 23577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:14.257601 23568 server_base.cc:1034] running on GCE node
I20260808 03:56:14.258224 23568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:14.259188 23568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:14.260406 23568 hybrid_clock.cc:648] HybridClock initialized: now 1786161374260344 us; error 52 us; skew 500 ppm
I20260808 03:56:14.260759 23568 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:14 dist-test-slave-jlzn krb5kdc[23220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161374, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:14.264122 23568 init.cc:377] Logged in from keytab as kudu/127.26.146.130@KRBTEST.COM (short username kudu)
I20260808 03:56:14.265606 23568 webserver.cc:460] Webserver started at http://127.26.146.130:34203/ using document root <none> and password file <none>
I20260808 03:56:14.266294 23568 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:14.266412 23568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:14.266708 23568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:14.269675 23568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "b1ca521443a44af1b39bd4ddd3c9e5af"
format_stamp: "Formatted at 2026-08-08 03:56:14 on dist-test-slave-jlzn"
server_key: "e9e8308be6658c9ae04a4ad666656101"
server_key_iv: "dad83f9ac82fa7caa7f98e0f7a5270be"
server_key_version: "encryptionkey@0"
I20260808 03:56:14.270429 23568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "b1ca521443a44af1b39bd4ddd3c9e5af"
format_stamp: "Formatted at 2026-08-08 03:56:14 on dist-test-slave-jlzn"
server_key: "e9e8308be6658c9ae04a4ad666656101"
server_key_iv: "dad83f9ac82fa7caa7f98e0f7a5270be"
server_key_version: "encryptionkey@0"
I20260808 03:56:14.275681 23568 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 03:56:14.279524 23591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:14.280921 23568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:56:14.281073 23568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "b1ca521443a44af1b39bd4ddd3c9e5af"
format_stamp: "Formatted at 2026-08-08 03:56:14 on dist-test-slave-jlzn"
server_key: "e9e8308be6658c9ae04a4ad666656101"
server_key_iv: "dad83f9ac82fa7caa7f98e0f7a5270be"
server_key_version: "encryptionkey@0"
I20260808 03:56:14.281263 23568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:14.309872 23568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:14.315351 23568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:14.315683 23568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:14.316978 23568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:56:14.318816 23568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:56:14.318918 23568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:14.319048 23568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:56:14.319108 23568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:14.349269 23568 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:46475
I20260808 03:56:14.349320 23753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:46475 every 8 connection(s)
I20260808 03:56:14.350512 23568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:56:14.360956 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 23568
I20260808 03:56:14.361156 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:56:14.361756 27210 external_mini_cluster.cc:1428] Setting key c3c21aa1cc4fa6b0ca6060fc4c4f4b2b
Aug 08 03:56:14 dist-test-slave-jlzn krb5kdc[23220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161374, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.130@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:56:14.370640 23756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:14.353003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:48715 (local address 127.26.146.190:45375)
0808 03:56:14.353451 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:14.353457 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:14.354398 (+ 941us) server_negotiation.cc:407] Connection header received
0808 03:56:14.356193 (+ 1795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:14.356197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:14.356264 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:14.356357 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:14.357772 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:14.358466 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:14.359934 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:14.360342 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:14.360431 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:14.364463 (+ 4032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:14.364470 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:14.364475 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:14.364515 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:14.367623 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:14.368411 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:14.368416 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:14.368418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:14.368573 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:14.368961 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:14.368965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:14.368968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:14.369293 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:14.369420 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:14.370078 (+ 658us) server_negotiation.cc:299] Negotiation successful
0808 03:56:14.370297 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":207,"threads_started":1}
I20260808 03:56:14.371470 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:14.353554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:45375 (local address 127.26.146.130:48715)
0808 03:56:14.354234 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:14.354283 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:14.355947 (+ 1664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:14.356549 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:14.356557 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:14.357080 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:14.357626 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:14.357643 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:14.358570 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:14.358574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:14.359796 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:14.359839 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:14.360514 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:14.360521 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:14.360825 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:14.361525 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:56:14.361553 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:14.364242 (+ 2689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:14.367776 (+ 3534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:14.367783 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:14.367796 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:14.368290 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:14.368670 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:14.368673 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:14.368675 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:14.368858 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:14.369404 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:14.369411 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:14.369740 (+ 329us) client_negotiation.cc:769] Sending connection context
0808 03:56:14.369955 (+ 215us) client_negotiation.cc:240] Negotiation successful
0808 03:56:14.370341 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":254,"threads_started":1}
I20260808 03:56:14.372958 23754 heartbeater.cc:347] Connected to a master server at 127.26.146.190:45375
I20260808 03:56:14.373274 23754 heartbeater.cc:464] Registering TS with master...
I20260808 03:56:14.374089 23754 heartbeater.cc:511] Master 127.26.146.190:45375 requested a full tablet report, sending...
I20260808 03:56:14.375681 23291 ts_manager.cc:194] Registered new tserver with Master: b1ca521443a44af1b39bd4ddd3c9e5af (127.26.146.130:46475)
I20260808 03:56:14.376705 23291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.130@KRBTEST.COM'} at 127.26.146.130:48715
WARNING: no policy specified for kudu/127.26.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
Entry for principal kudu/127.26.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab.
I20260808 03:56:14.428591 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:45375
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:36991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:14.761788 23761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:14.762082 23761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:14.762184 23761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:14.769356 23761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:14.769469 23761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:14.769585 23761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:56:14.779392 23761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:36991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.131.keytab
--principal=kudu/127.26.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:45375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:14.780622 23761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:14.781863 23761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:14.794669 23769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:14.794759 23771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:14.794732 23773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:14.795512 23761 server_base.cc:1034] running on GCE node
I20260808 03:56:14.796172 23761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:14.797041 23761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:14.798254 23761 hybrid_clock.cc:648] HybridClock initialized: now 1786161374798204 us; error 40 us; skew 500 ppm
I20260808 03:56:14.798548 23761 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:14 dist-test-slave-jlzn krb5kdc[23220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161374, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:14.801743 23761 init.cc:377] Logged in from keytab as kudu/127.26.146.131@KRBTEST.COM (short username kudu)
I20260808 03:56:14.803169 23761 webserver.cc:460] Webserver started at http://127.26.146.131:44299/ using document root <none> and password file <none>
I20260808 03:56:14.803856 23761 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:14.803982 23761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:14.804283 23761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:14.807101 23761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "d7b015b9d68e4c21b204918fe6b3014e"
format_stamp: "Formatted at 2026-08-08 03:56:14 on dist-test-slave-jlzn"
server_key: "382b5a6955f8e40c180df33274e1fc82"
server_key_iv: "544c37078b638f5cf7e9820b350e2516"
server_key_version: "encryptionkey@0"
I20260808 03:56:14.807983 23761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "d7b015b9d68e4c21b204918fe6b3014e"
format_stamp: "Formatted at 2026-08-08 03:56:14 on dist-test-slave-jlzn"
server_key: "382b5a6955f8e40c180df33274e1fc82"
server_key_iv: "544c37078b638f5cf7e9820b350e2516"
server_key_version: "encryptionkey@0"
I20260808 03:56:14.813028 23761 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 03:56:14.816424 23784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:14.817787 23761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:56:14.817962 23761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "d7b015b9d68e4c21b204918fe6b3014e"
format_stamp: "Formatted at 2026-08-08 03:56:14 on dist-test-slave-jlzn"
server_key: "382b5a6955f8e40c180df33274e1fc82"
server_key_iv: "544c37078b638f5cf7e9820b350e2516"
server_key_version: "encryptionkey@0"
I20260808 03:56:14.818182 23761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:14.841326 23761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:14.841665 23561 heartbeater.cc:503] Master 127.26.146.190:45375 was elected leader, sending a full tablet report...
I20260808 03:56:14.845883 23761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:14.846133 23761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:14.847047 23761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:56:14.848387 23761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:56:14.848460 23761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:14.848552 23761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:56:14.848613 23761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:14.883548 23761 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:42767
I20260808 03:56:14.883625 23939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:42767 every 8 connection(s)
I20260808 03:56:14.884878 23761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:56:14.889739 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 23761
I20260808 03:56:14.889914 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:56:14.890374 27210 external_mini_cluster.cc:1428] Setting key 120170437fd2ce263227d9185ecbd6a8
Aug 08 03:56:14 dist-test-slave-jlzn krb5kdc[23220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161374, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.131@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:56:14.904881 23946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:14.887657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:39603 (local address 127.26.146.190:45375)
0808 03:56:14.888168 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:14.888172 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:14.889091 (+ 919us) server_negotiation.cc:407] Connection header received
0808 03:56:14.891260 (+ 2169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:14.891266 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:14.891354 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:14.891445 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:14.893190 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:14.894040 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:14.895566 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:14.895993 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:14.896105 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:14.899370 (+ 3265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:14.899374 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:14.899378 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:14.899416 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:14.901903 (+ 2487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:14.902580 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:14.902584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:14.902586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:14.902706 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:14.903119 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:14.903125 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:14.903127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:14.903405 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:14.903551 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:14.904273 (+ 722us) server_negotiation.cc:299] Negotiation successful
0808 03:56:14.904498 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":264,"threads_started":1}
I20260808 03:56:14.905771 23947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:14.888113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:45375 (local address 127.26.146.131:39603)
0808 03:56:14.888876 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:14.888923 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:14.890954 (+ 2031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:14.891730 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:14.891739 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:14.892311 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:14.892972 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:14.892991 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:14.894184 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:14.894188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:14.895420 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:14.895433 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:14.896204 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:14.896212 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:14.896440 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:14.897072 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:56:14.897095 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:14.899177 (+ 2082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:14.902042 (+ 2865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:14.902050 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:14.902067 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:14.902464 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:14.902808 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:14.902813 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:14.902816 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:14.903006 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:14.903532 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:14.903540 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:14.903889 (+ 349us) client_negotiation.cc:769] Sending connection context
0808 03:56:14.904142 (+ 253us) client_negotiation.cc:240] Negotiation successful
0808 03:56:14.904648 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":297,"threads_started":1}
I20260808 03:56:14.907063 23942 heartbeater.cc:347] Connected to a master server at 127.26.146.190:45375
I20260808 03:56:14.907326 23942 heartbeater.cc:464] Registering TS with master...
I20260808 03:56:14.908041 23942 heartbeater.cc:511] Master 127.26.146.190:45375 requested a full tablet report, sending...
I20260808 03:56:14.909400 23291 ts_manager.cc:194] Registered new tserver with Master: d7b015b9d68e4c21b204918fe6b3014e (127.26.146.131:42767)
I20260808 03:56:14.910190 23291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.146.131@KRBTEST.COM'} at 127.26.146.131:39603
I20260808 03:56:14.917590 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:56:14.933789 23946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:14.921685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55776 (local address 127.26.146.190:45375)
0808 03:56:14.921873 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:14.921879 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:14.922287 (+ 408us) server_negotiation.cc:407] Connection header received
0808 03:56:14.922425 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:14.922429 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:14.922491 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:14.922579 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:14.923924 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:14.924664 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:14.925895 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:14.926315 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:14.926408 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:14.928932 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:14.928937 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:14.928940 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:14.928972 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:14.931257 (+ 2285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:14.931988 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:14.931993 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:14.931995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:14.932131 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:14.932486 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:14.932490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:14.932492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:14.932862 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:14.933052 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:14.933348 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 03:56:14.933525 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 03:56:14.937489 23291 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55776:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:56:14.944391 23291 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:56:14.966943 23960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:14.956262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:45813 (local address 127.0.0.1:33002)
0808 03:56:14.957115 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:14.957167 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:14.957342 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:14.958505 (+ 1163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:14.958513 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:14.958560 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:14.958995 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:14.959008 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:14.961305 (+ 2297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:14.961311 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:14.963846 (+ 2535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:14.963864 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:14.965915 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:14.965922 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:14.966134 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:14.966191 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 03:56:14.966302 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 03:56:14.966497 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":537,"threads_started":1}
I20260808 03:56:14.968753 23961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:14.956424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33002 (local address 127.26.146.129:45813)
0808 03:56:14.957884 (+ 1460us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:14.957891 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:14.957913 (+ 22us) server_negotiation.cc:407] Connection header received
0808 03:56:14.958012 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:14.958019 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:14.958253 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:14.959220 (+ 967us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:14.959456 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:14.961138 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:14.964070 (+ 2932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:14.965658 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:14.967908 (+ 2250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:14.968116 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:14.968189 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 03:56:14.968369 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1325,"thread_start_us":1196,"threads_started":1}
I20260808 03:56:14.972680 23469 tablet_service.cc:1467] Processing CreateTablet for tablet 8fd80757ed4f45248e6692f260d3c237 (DEFAULT_TABLE table=test-table [id=d3fe0d623adb4988a731fc5551e24393]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:56:14.974100 23469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fd80757ed4f45248e6692f260d3c237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:14.975559 23960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:14.957640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:46475 (local address 127.0.0.1:37798)
0808 03:56:14.967214 (+ 9574us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:14.967232 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:14.967332 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:14.969448 (+ 2116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:14.969452 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:14.969477 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:14.969694 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:14.969706 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:14.971574 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:14.971580 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:14.973405 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:14.973416 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:14.975021 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:14.975027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:14.975151 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:14.975178 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 03:56:14.975228 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 03:56:14.975334 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9476,"thread_start_us":9685,"threads_started":1}
I20260808 03:56:14.975920 23965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:14.956752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:42767 (local address 127.0.0.1:47804)
0808 03:56:14.967106 (+ 10354us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:14.967127 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:14.967251 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:14.969084 (+ 1833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:14.969089 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:14.969114 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:14.969438 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:14.969450 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:14.971138 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:14.971143 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:14.973350 (+ 2207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:14.973366 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:14.975089 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:14.975096 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:14.975281 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:14.975315 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 03:56:14.975383 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 03:56:14.975512 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10216,"thread_start_us":11080,"threads_started":1}
I20260808 03:56:14.976454 23967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:14.958132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37798 (local address 127.26.146.130:46475)
0808 03:56:14.968902 (+ 10770us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:14.968911 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:14.968938 (+ 27us) server_negotiation.cc:407] Connection header received
0808 03:56:14.969029 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:14.969036 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:14.969257 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:14.969435 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:14.969850 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:14.971346 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:14.973567 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:14.974829 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:14.975835 (+ 1006us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:14.976003 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:14.976073 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 03:56:14.976237 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10611,"thread_start_us":10518,"threads_started":1}
I20260808 03:56:14.976524 23963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:14.957399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47804 (local address 127.26.146.131:42767)
0808 03:56:14.968476 (+ 11077us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:14.968485 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:14.968517 (+ 32us) server_negotiation.cc:407] Connection header received
0808 03:56:14.968623 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:14.968632 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:14.968873 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:14.969081 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:14.969662 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:14.970968 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:14.973514 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:14.974847 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:14.975726 (+ 879us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:14.975935 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:14.976006 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 03:56:14.976175 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10886,"thread_start_us":10941,"threads_started":1}
I20260808 03:56:14.980392 23850 tablet_service.cc:1467] Processing CreateTablet for tablet 8fd80757ed4f45248e6692f260d3c237 (DEFAULT_TABLE table=test-table [id=d3fe0d623adb4988a731fc5551e24393]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:56:14.981900 23850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fd80757ed4f45248e6692f260d3c237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:14.981848 23661 tablet_service.cc:1467] Processing CreateTablet for tablet 8fd80757ed4f45248e6692f260d3c237 (DEFAULT_TABLE table=test-table [id=d3fe0d623adb4988a731fc5551e24393]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:56:14.982870 23661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fd80757ed4f45248e6692f260d3c237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:14.990327 23968 tablet_bootstrap.cc:492] T 8fd80757ed4f45248e6692f260d3c237 P b84eceec70ae4e28aec897a5ee80dcc4: Bootstrap starting.
I20260808 03:56:14.993984 23968 tablet_bootstrap.cc:654] T 8fd80757ed4f45248e6692f260d3c237 P b84eceec70ae4e28aec897a5ee80dcc4: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:14.995039 23969 tablet_bootstrap.cc:492] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af: Bootstrap starting.
I20260808 03:56:14.995579 23968 log.cc:824] T 8fd80757ed4f45248e6692f260d3c237 P b84eceec70ae4e28aec897a5ee80dcc4: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:14.998720 23969 tablet_bootstrap.cc:654] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:15.000058 23969 log.cc:824] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:15.000716 23970 tablet_bootstrap.cc:492] T 8fd80757ed4f45248e6692f260d3c237 P d7b015b9d68e4c21b204918fe6b3014e: Bootstrap starting.
I20260808 03:56:15.003921 23968 tablet_bootstrap.cc:492] T 8fd80757ed4f45248e6692f260d3c237 P b84eceec70ae4e28aec897a5ee80dcc4: No bootstrap required, opened a new log
I20260808 03:56:15.004078 23970 tablet_bootstrap.cc:654] T 8fd80757ed4f45248e6692f260d3c237 P d7b015b9d68e4c21b204918fe6b3014e: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:15.004124 23969 tablet_bootstrap.cc:492] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af: No bootstrap required, opened a new log
I20260808 03:56:15.004192 23968 ts_tablet_manager.cc:1397] T 8fd80757ed4f45248e6692f260d3c237 P b84eceec70ae4e28aec897a5ee80dcc4: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.004s
I20260808 03:56:15.004365 23969 ts_tablet_manager.cc:1397] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260808 03:56:15.005303 23970 log.cc:824] T 8fd80757ed4f45248e6692f260d3c237 P d7b015b9d68e4c21b204918fe6b3014e: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:15.008481 23970 tablet_bootstrap.cc:492] T 8fd80757ed4f45248e6692f260d3c237 P d7b015b9d68e4c21b204918fe6b3014e: No bootstrap required, opened a new log
I20260808 03:56:15.008738 23970 ts_tablet_manager.cc:1397] T 8fd80757ed4f45248e6692f260d3c237 P d7b015b9d68e4c21b204918fe6b3014e: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.001s
I20260808 03:56:15.011740 23968 raft_consensus.cc:348] T 8fd80757ed4f45248e6692f260d3c237 P b84eceec70ae4e28aec897a5ee80dcc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84eceec70ae4e28aec897a5ee80dcc4" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 45813 } } peers { permanent_uuid: "b1ca521443a44af1b39bd4ddd3c9e5af" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 46475 } } peers { permanent_uuid: "d7b015b9d68e4c21b204918fe6b3014e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 42767 } }
I20260808 03:56:15.012133 23968 raft_consensus.cc:374] T 8fd80757ed4f45248e6692f260d3c237 P b84eceec70ae4e28aec897a5ee80dcc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:15.012233 23968 raft_consensus.cc:729] T 8fd80757ed4f45248e6692f260d3c237 P b84eceec70ae4e28aec897a5ee80dcc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b84eceec70ae4e28aec897a5ee80dcc4, State: Initialized, Role: FOLLOWER
I20260808 03:56:15.012799 23968 consensus_queue.cc:260] T 8fd80757ed4f45248e6692f260d3c237 P b84eceec70ae4e28aec897a5ee80dcc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84eceec70ae4e28aec897a5ee80dcc4" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 45813 } } peers { permanent_uuid: "b1ca521443a44af1b39bd4ddd3c9e5af" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 46475 } } peers { permanent_uuid: "d7b015b9d68e4c21b204918fe6b3014e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 42767 } }
I20260808 03:56:15.014009 23969 raft_consensus.cc:348] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84eceec70ae4e28aec897a5ee80dcc4" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 45813 } } peers { permanent_uuid: "b1ca521443a44af1b39bd4ddd3c9e5af" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 46475 } } peers { permanent_uuid: "d7b015b9d68e4c21b204918fe6b3014e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 42767 } }
I20260808 03:56:15.014418 23969 raft_consensus.cc:374] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:15.014458 23968 ts_tablet_manager.cc:1428] T 8fd80757ed4f45248e6692f260d3c237 P b84eceec70ae4e28aec897a5ee80dcc4: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260808 03:56:15.014500 23969 raft_consensus.cc:729] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1ca521443a44af1b39bd4ddd3c9e5af, State: Initialized, Role: FOLLOWER
I20260808 03:56:15.015192 23969 consensus_queue.cc:260] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84eceec70ae4e28aec897a5ee80dcc4" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 45813 } } peers { permanent_uuid: "b1ca521443a44af1b39bd4ddd3c9e5af" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 46475 } } peers { permanent_uuid: "d7b015b9d68e4c21b204918fe6b3014e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 42767 } }
I20260808 03:56:15.016294 23754 heartbeater.cc:503] Master 127.26.146.190:45375 was elected leader, sending a full tablet report...
I20260808 03:56:15.016878 23969 ts_tablet_manager.cc:1428] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af: Time spent starting tablet: real 0.012s user 0.007s sys 0.005s
I20260808 03:56:15.019614 23970 raft_consensus.cc:348] T 8fd80757ed4f45248e6692f260d3c237 P d7b015b9d68e4c21b204918fe6b3014e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84eceec70ae4e28aec897a5ee80dcc4" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 45813 } } peers { permanent_uuid: "b1ca521443a44af1b39bd4ddd3c9e5af" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 46475 } } peers { permanent_uuid: "d7b015b9d68e4c21b204918fe6b3014e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 42767 } }
I20260808 03:56:15.020069 23970 raft_consensus.cc:374] T 8fd80757ed4f45248e6692f260d3c237 P d7b015b9d68e4c21b204918fe6b3014e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:15.020175 23970 raft_consensus.cc:729] T 8fd80757ed4f45248e6692f260d3c237 P d7b015b9d68e4c21b204918fe6b3014e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7b015b9d68e4c21b204918fe6b3014e, State: Initialized, Role: FOLLOWER
I20260808 03:56:15.020704 23970 consensus_queue.cc:260] T 8fd80757ed4f45248e6692f260d3c237 P d7b015b9d68e4c21b204918fe6b3014e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84eceec70ae4e28aec897a5ee80dcc4" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 45813 } } peers { permanent_uuid: "b1ca521443a44af1b39bd4ddd3c9e5af" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 46475 } } peers { permanent_uuid: "d7b015b9d68e4c21b204918fe6b3014e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 42767 } }
I20260808 03:56:15.021731 23942 heartbeater.cc:503] Master 127.26.146.190:45375 was elected leader, sending a full tablet report...
I20260808 03:56:15.022320 23970 ts_tablet_manager.cc:1428] T 8fd80757ed4f45248e6692f260d3c237 P d7b015b9d68e4c21b204918fe6b3014e: Time spent starting tablet: real 0.013s user 0.010s sys 0.003s
W20260808 03:56:15.061474 23562 tablet.cc:2369] T 8fd80757ed4f45248e6692f260d3c237 P b84eceec70ae4e28aec897a5ee80dcc4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:56:15.102398 23755 tablet.cc:2369] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:56:15.136582 23944 tablet.cc:2369] T 8fd80757ed4f45248e6692f260d3c237 P d7b015b9d68e4c21b204918fe6b3014e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:56:15.333114 23977 raft_consensus.cc:482] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:56:15.333428 23977 raft_consensus.cc:504] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84eceec70ae4e28aec897a5ee80dcc4" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 45813 } } peers { permanent_uuid: "b1ca521443a44af1b39bd4ddd3c9e5af" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 46475 } } peers { permanent_uuid: "d7b015b9d68e4c21b204918fe6b3014e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 42767 } }
I20260808 03:56:15.335219 23977 leader_election.cc:288] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b84eceec70ae4e28aec897a5ee80dcc4 (127.26.146.129:45813), d7b015b9d68e4c21b204918fe6b3014e (127.26.146.131:42767)
I20260808 03:56:15.341853 23961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:15.335575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:34555 (local address 127.26.146.129:45813)
0808 03:56:15.335794 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:15.335821 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:15.336362 (+ 541us) server_negotiation.cc:407] Connection header received
0808 03:56:15.336573 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:15.336580 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:15.336653 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:15.336735 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:15.337343 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:15.338102 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:15.340075 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:15.340915 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:15.341071 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:15.341148 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:15.341479 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 03:56:15.341606 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 03:56:15.341863 23963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:15.336058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:42339 (local address 127.26.146.131:42767)
0808 03:56:15.336241 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:15.336246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:15.336587 (+ 341us) server_negotiation.cc:407] Connection header received
0808 03:56:15.336741 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:15.336746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:15.336821 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:15.336918 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:15.337482 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:15.338191 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:15.340163 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:15.340960 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:15.341118 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:15.341201 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:15.341500 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 03:56:15.341622 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 03:56:15.342038 23980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:15.335405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:45813 (local address 127.26.146.130:34555)
0808 03:56:15.336239 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:15.336260 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:15.336413 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:15.336761 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:15.336766 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:15.336788 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:15.337197 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:15.337209 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:15.338228 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:15.338233 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:15.339927 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:15.339943 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:15.341153 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:15.341161 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:15.341330 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:15.341355 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 03:56:15.341429 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 03:56:15.341545 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"mutex_wait_us":2,"thread_start_us":457,"threads_started":1}
I20260808 03:56:15.342286 23981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:15.335645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:42767 (local address 127.26.146.130:42339)
0808 03:56:15.336477 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:15.336493 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:15.336624 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:15.336945 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:15.336948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:15.336968 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:15.337331 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:15.337342 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:15.338335 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:15.338338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:15.339994 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:15.340016 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:15.341198 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:15.341205 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:15.341344 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:15.341369 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 03:56:15.341456 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 03:56:15.341591 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"mutex_wait_us":1,"thread_start_us":499,"threads_started":1}
I20260808 03:56:15.342955 23494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fd80757ed4f45248e6692f260d3c237" candidate_uuid: "b1ca521443a44af1b39bd4ddd3c9e5af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b84eceec70ae4e28aec897a5ee80dcc4" is_pre_election: true
I20260808 03:56:15.343314 23876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fd80757ed4f45248e6692f260d3c237" candidate_uuid: "b1ca521443a44af1b39bd4ddd3c9e5af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7b015b9d68e4c21b204918fe6b3014e" is_pre_election: true
I20260808 03:56:15.343585 23494 raft_consensus.cc:2454] T 8fd80757ed4f45248e6692f260d3c237 P b84eceec70ae4e28aec897a5ee80dcc4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1ca521443a44af1b39bd4ddd3c9e5af in term 0.
I20260808 03:56:15.343839 23876 raft_consensus.cc:2454] T 8fd80757ed4f45248e6692f260d3c237 P d7b015b9d68e4c21b204918fe6b3014e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1ca521443a44af1b39bd4ddd3c9e5af in term 0.
I20260808 03:56:15.344347 23593 leader_election.cc:302] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1ca521443a44af1b39bd4ddd3c9e5af, b84eceec70ae4e28aec897a5ee80dcc4; no voters:
I20260808 03:56:15.344831 23977 raft_consensus.cc:2781] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:56:15.344985 23977 raft_consensus.cc:482] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:56:15.345127 23977 raft_consensus.cc:3037] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:15.347211 23977 raft_consensus.cc:504] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84eceec70ae4e28aec897a5ee80dcc4" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 45813 } } peers { permanent_uuid: "b1ca521443a44af1b39bd4ddd3c9e5af" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 46475 } } peers { permanent_uuid: "d7b015b9d68e4c21b204918fe6b3014e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 42767 } }
I20260808 03:56:15.347846 23977 leader_election.cc:288] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af [CANDIDATE]: Term 1 election: Requested vote from peers b84eceec70ae4e28aec897a5ee80dcc4 (127.26.146.129:45813), d7b015b9d68e4c21b204918fe6b3014e (127.26.146.131:42767)
I20260808 03:56:15.348178 23494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fd80757ed4f45248e6692f260d3c237" candidate_uuid: "b1ca521443a44af1b39bd4ddd3c9e5af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b84eceec70ae4e28aec897a5ee80dcc4"
I20260808 03:56:15.348347 23494 raft_consensus.cc:3037] T 8fd80757ed4f45248e6692f260d3c237 P b84eceec70ae4e28aec897a5ee80dcc4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:15.348306 23876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fd80757ed4f45248e6692f260d3c237" candidate_uuid: "b1ca521443a44af1b39bd4ddd3c9e5af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7b015b9d68e4c21b204918fe6b3014e"
I20260808 03:56:15.348487 23876 raft_consensus.cc:3037] T 8fd80757ed4f45248e6692f260d3c237 P d7b015b9d68e4c21b204918fe6b3014e [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:15.350535 23494 raft_consensus.cc:2454] T 8fd80757ed4f45248e6692f260d3c237 P b84eceec70ae4e28aec897a5ee80dcc4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1ca521443a44af1b39bd4ddd3c9e5af in term 1.
I20260808 03:56:15.350880 23593 leader_election.cc:302] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1ca521443a44af1b39bd4ddd3c9e5af, b84eceec70ae4e28aec897a5ee80dcc4; no voters:
I20260808 03:56:15.351173 23977 raft_consensus.cc:2781] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:15.351524 23876 raft_consensus.cc:2454] T 8fd80757ed4f45248e6692f260d3c237 P d7b015b9d68e4c21b204918fe6b3014e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1ca521443a44af1b39bd4ddd3c9e5af in term 1.
I20260808 03:56:15.351645 23977 raft_consensus.cc:686] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af [term 1 LEADER]: Becoming Leader. State: Replica: b1ca521443a44af1b39bd4ddd3c9e5af, State: Running, Role: LEADER
I20260808 03:56:15.352322 23977 consensus_queue.cc:237] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84eceec70ae4e28aec897a5ee80dcc4" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 45813 } } peers { permanent_uuid: "b1ca521443a44af1b39bd4ddd3c9e5af" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 46475 } } peers { permanent_uuid: "d7b015b9d68e4c21b204918fe6b3014e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 42767 } }
I20260808 03:56:15.356616 23290 catalog_manager.cc:5495] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af reported cstate change: term changed from 0 to 1, leader changed from <none> to b1ca521443a44af1b39bd4ddd3c9e5af (127.26.146.130). New cstate: current_term: 1 leader_uuid: "b1ca521443a44af1b39bd4ddd3c9e5af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84eceec70ae4e28aec897a5ee80dcc4" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 45813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1ca521443a44af1b39bd4ddd3c9e5af" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 46475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7b015b9d68e4c21b204918fe6b3014e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 42767 } health_report { overall_health: UNKNOWN } } }
I20260808 03:56:15.431746 23967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:15.426691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37808 (local address 127.26.146.130:46475)
0808 03:56:15.426889 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:15.426893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:15.426910 (+ 17us) server_negotiation.cc:407] Connection header received
0808 03:56:15.427010 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:15.427014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:15.427092 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:15.427212 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:56:15.427546 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:15.428368 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:15.429776 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:15.430195 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:15.430288 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:15.430707 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:56:15.431216 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:56:15.431325 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:15.431382 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 03:56:15.431447 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 03:56:15.440230 23494 raft_consensus.cc:1261] T 8fd80757ed4f45248e6692f260d3c237 P b84eceec70ae4e28aec897a5ee80dcc4 [term 1 FOLLOWER]: Refusing update from remote peer b1ca521443a44af1b39bd4ddd3c9e5af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:56:15.440230 23876 raft_consensus.cc:1261] T 8fd80757ed4f45248e6692f260d3c237 P d7b015b9d68e4c21b204918fe6b3014e [term 1 FOLLOWER]: Refusing update from remote peer b1ca521443a44af1b39bd4ddd3c9e5af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:56:15.441267 23983 consensus_queue.cc:1035] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7b015b9d68e4c21b204918fe6b3014e" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 42767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:56:15.441691 23977 consensus_queue.cc:1035] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af [LEADER]: Connected to new peer: Peer: permanent_uuid: "b84eceec70ae4e28aec897a5ee80dcc4" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 45813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:56:15.452720 23988 mvcc.cc:205] Tried to move back new op lower bound from 7316116993791852544 to 7316116993447047168. Current Snapshot: MvccSnapshot[applied={T|T < 7316116993791852544}]
I20260808 03:56:15.464802 23291 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:56:15.465159 23291 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:56:15.468614 23291 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ee3ab494fb34e4da419f000e3ebb2d0: Sending DeleteTablet for 3 replicas of tablet 8fd80757ed4f45248e6692f260d3c237
I20260808 03:56:15.469740 23850 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fd80757ed4f45248e6692f260d3c237 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:56:15 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:47804
I20260808 03:56:15.469762 23661 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fd80757ed4f45248e6692f260d3c237 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:56:15 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:37798
I20260808 03:56:15.470753 23469 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fd80757ed4f45248e6692f260d3c237 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:56:15 UTC) from {username='kudu', principal='kudu/127.26.146.190@KRBTEST.COM'} at 127.0.0.1:33002
I20260808 03:56:15.472007 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 23383
I20260808 03:56:15.476928 24005 tablet_replica.cc:330] T 8fd80757ed4f45248e6692f260d3c237 P b84eceec70ae4e28aec897a5ee80dcc4: stopping tablet replica
I20260808 03:56:15.477406 24005 raft_consensus.cc:2229] T 8fd80757ed4f45248e6692f260d3c237 P b84eceec70ae4e28aec897a5ee80dcc4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:56:15.477912 24005 raft_consensus.cc:2258] T 8fd80757ed4f45248e6692f260d3c237 P b84eceec70ae4e28aec897a5ee80dcc4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:56:15.482157 24003 tablet_replica.cc:330] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af: stopping tablet replica
I20260808 03:56:15.482472 24003 raft_consensus.cc:2229] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:56:15.482959 24003 raft_consensus.cc:2258] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:56:15.483687 24005 ts_tablet_manager.cc:1905] T 8fd80757ed4f45248e6692f260d3c237 P b84eceec70ae4e28aec897a5ee80dcc4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:56:15.492926 24003 ts_tablet_manager.cc:1905] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:56:15.494299 24005 ts_tablet_manager.cc:1918] T 8fd80757ed4f45248e6692f260d3c237 P b84eceec70ae4e28aec897a5ee80dcc4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:56:15.494550 24005 log.cc:1196] T 8fd80757ed4f45248e6692f260d3c237 P b84eceec70ae4e28aec897a5ee80dcc4: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-0/wal/wals/8fd80757ed4f45248e6692f260d3c237
I20260808 03:56:15.495136 24005 ts_tablet_manager.cc:1939] T 8fd80757ed4f45248e6692f260d3c237 P b84eceec70ae4e28aec897a5ee80dcc4: Deleting consensus metadata
I20260808 03:56:15.495755 24001 tablet_replica.cc:330] T 8fd80757ed4f45248e6692f260d3c237 P d7b015b9d68e4c21b204918fe6b3014e: stopping tablet replica
I20260808 03:56:15.496213 24001 raft_consensus.cc:2229] T 8fd80757ed4f45248e6692f260d3c237 P d7b015b9d68e4c21b204918fe6b3014e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:56:15.496802 24001 raft_consensus.cc:2258] T 8fd80757ed4f45248e6692f260d3c237 P d7b015b9d68e4c21b204918fe6b3014e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:56:15.498044 23265 catalog_manager.cc:4841] TS b84eceec70ae4e28aec897a5ee80dcc4 (127.26.146.129:45813): tablet 8fd80757ed4f45248e6692f260d3c237 (table test-table [id=d3fe0d623adb4988a731fc5551e24393]) successfully deleted
Aug 08 03:56:15 dist-test-slave-jlzn krb5kdc[23220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.129@KRBTEST.COM
I20260808 03:56:15.499791 24003 ts_tablet_manager.cc:1918] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:56:15.499974 24003 log.cc:1196] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/8fd80757ed4f45248e6692f260d3c237
I20260808 03:56:15.500530 24003 ts_tablet_manager.cc:1939] T 8fd80757ed4f45248e6692f260d3c237 P b1ca521443a44af1b39bd4ddd3c9e5af: Deleting consensus metadata
I20260808 03:56:15.500751 24001 ts_tablet_manager.cc:1905] T 8fd80757ed4f45248e6692f260d3c237 P d7b015b9d68e4c21b204918fe6b3014e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:56:15.502270 23265 catalog_manager.cc:4841] TS b1ca521443a44af1b39bd4ddd3c9e5af (127.26.146.130:46475): tablet 8fd80757ed4f45248e6692f260d3c237 (table test-table [id=d3fe0d623adb4988a731fc5551e24393]) successfully deleted
I20260808 03:56:15.507191 24001 ts_tablet_manager.cc:1918] T 8fd80757ed4f45248e6692f260d3c237 P d7b015b9d68e4c21b204918fe6b3014e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:56:15.507423 24001 log.cc:1196] T 8fd80757ed4f45248e6692f260d3c237 P d7b015b9d68e4c21b204918fe6b3014e: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/8fd80757ed4f45248e6692f260d3c237
I20260808 03:56:15.507509 23961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:15.473538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33004 (local address 127.26.146.129:45813)
0808 03:56:15.473742 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:15.473748 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:15.487828 (+ 14080us) server_negotiation.cc:407] Connection header received
0808 03:56:15.488140 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:15.488147 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:15.488243 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:15.488335 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:15.490274 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:15.491523 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:15.493609 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:15.494292 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:15.494452 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:15.500424 (+ 5972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:15.500439 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:15.500457 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:15.500505 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:15.504200 (+ 3695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:15.505125 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:15.505134 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:15.505138 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:15.505310 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:15.505740 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:15.505746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:15.505749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:15.506259 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:15.506579 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:15.506920 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 03:56:15.507140 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 03:56:15.508157 24001 ts_tablet_manager.cc:1939] T 8fd80757ed4f45248e6692f260d3c237 P d7b015b9d68e4c21b204918fe6b3014e: Deleting consensus metadata
I20260808 03:56:15.508275 23552 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:15.509670 23267 catalog_manager.cc:4841] TS d7b015b9d68e4c21b204918fe6b3014e (127.26.146.131:42767): tablet 8fd80757ed4f45248e6692f260d3c237 (table test-table [id=d3fe0d623adb4988a731fc5551e24393]) successfully deleted
I20260808 03:56:15.598816 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 23383
I20260808 03:56:15.630641 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 23568
Aug 08 03:56:15 dist-test-slave-jlzn krb5kdc[23220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.130@KRBTEST.COM
I20260808 03:56:15.650812 23967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:15.631726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37822 (local address 127.26.146.130:46475)
0808 03:56:15.631935 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:15.631940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:15.631966 (+ 26us) server_negotiation.cc:407] Connection header received
0808 03:56:15.632146 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:15.632156 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:15.632266 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:15.632613 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:15.634980 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:15.636875 (+ 1895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:15.638935 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:15.639643 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:15.639755 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:15.644827 (+ 5072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:15.644842 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:15.644858 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:15.644896 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:15.648083 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:15.648750 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:15.648758 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:15.648760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:15.648885 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:15.649257 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:15.649261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:15.649263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:15.649644 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:15.649967 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:15.650251 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 03:56:15.650467 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 03:56:15.651559 23748 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:15.763671 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 23568
I20260808 03:56:15.789299 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 23761
Aug 08 03:56:15 dist-test-slave-jlzn krb5kdc[23220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.131@KRBTEST.COM
I20260808 03:56:15.804461 23963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:15.790116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47812 (local address 127.26.146.131:42767)
0808 03:56:15.790314 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:15.790319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:15.790338 (+ 19us) server_negotiation.cc:407] Connection header received
0808 03:56:15.790497 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:15.790501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:15.790575 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:15.790660 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:15.792160 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:15.792883 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:15.794132 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:15.794581 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:15.794690 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:15.799038 (+ 4348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:15.799075 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:15.799093 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:15.799130 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:15.801791 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:15.802424 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:15.802431 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:15.802434 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:15.802588 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:15.802969 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:15.802973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:15.802975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:15.803340 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:15.803558 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:15.803902 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 03:56:15.804099 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 03:56:15.805146 23931 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:15.892588 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 23761
I20260808 03:56:15.912704 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 23235
I20260808 03:56:15.913854 23336 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:16.022289 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 23235
2026-08-08T03:56:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4805 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T03:56:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:56:16Z Disabled control of system clock
I20260808 03:56:16.113713 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:36407
--webserver_interface=127.26.146.190
--webserver_port=0
--builtin_ntp_servers=127.26.146.148:36833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:36407
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 03:56:16.445955 24027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:16.446234 24027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:16.446327 24027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:16.453438 24027 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:16.453538 24027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:16.453583 24027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:16.453637 24027 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:16.453680 24027 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:16.462541 24027 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:36833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:36407
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:36407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:16.463727 24027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:16.465508 24027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:16.476506 24033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:16.476570 24036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:16.476611 24034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:16.477722 24027 server_base.cc:1034] running on GCE node
I20260808 03:56:16.478464 24027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:16.479770 24027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:16.481006 24027 hybrid_clock.cc:648] HybridClock initialized: now 1786161376480949 us; error 40 us; skew 500 ppm
I20260808 03:56:16.481340 24027 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:56:16.483528 24027 webserver.cc:460] Webserver started at http://127.26.146.190:45175/ using document root <none> and password file <none>
I20260808 03:56:16.484216 24027 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:16.484308 24027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:16.484607 24027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:16.487468 24027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "c963efac28784c00b5ad30c7dfb7ef02"
format_stamp: "Formatted at 2026-08-08 03:56:16 on dist-test-slave-jlzn"
server_key: "f64d781b91d16e74a6737ff344dc799b"
server_key_iv: "d85595b2880d9a7b6b59fc2693b14820"
server_key_version: "encryptionkey@0"
I20260808 03:56:16.488348 24027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "c963efac28784c00b5ad30c7dfb7ef02"
format_stamp: "Formatted at 2026-08-08 03:56:16 on dist-test-slave-jlzn"
server_key: "f64d781b91d16e74a6737ff344dc799b"
server_key_iv: "d85595b2880d9a7b6b59fc2693b14820"
server_key_version: "encryptionkey@0"
I20260808 03:56:16.493463 24027 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 03:56:16.496872 24049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:16.498109 24027 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 03:56:16.498267 24027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "c963efac28784c00b5ad30c7dfb7ef02"
format_stamp: "Formatted at 2026-08-08 03:56:16 on dist-test-slave-jlzn"
server_key: "f64d781b91d16e74a6737ff344dc799b"
server_key_iv: "d85595b2880d9a7b6b59fc2693b14820"
server_key_version: "encryptionkey@0"
I20260808 03:56:16.498467 24027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:16.515000 24027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:16.515877 24027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:16.516163 24027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:16.533658 24027 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:36407
I20260808 03:56:16.533677 24118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:36407 every 8 connection(s)
I20260808 03:56:16.534957 24027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:56:16.538749 24119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:16.541728 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 24027
I20260808 03:56:16.541906 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:56:16.542486 27210 external_mini_cluster.cc:1428] Setting key dc675231bbfb445e8c5955d96ef653b1
I20260808 03:56:16.550503 24119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c963efac28784c00b5ad30c7dfb7ef02: Bootstrap starting.
I20260808 03:56:16.553558 24119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c963efac28784c00b5ad30c7dfb7ef02: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:16.554651 24119 log.cc:824] T 00000000000000000000000000000000 P c963efac28784c00b5ad30c7dfb7ef02: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:16.555663 24124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:16.544512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51882 (local address 127.26.146.190:36407)
0808 03:56:16.545452 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:16.545463 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:16.545503 (+ 40us) server_negotiation.cc:407] Connection header received
0808 03:56:16.547189 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:16.547225 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:16.547782 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:16.548181 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:16.548745 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:16.549963 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:16.551304 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:16.552050 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:16.552229 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:16.553279 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:16.553287 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:16.553300 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:16.553842 (+ 542us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:16.553940 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:16.553964 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:16.554064 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:16.554282 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 03:56:16.554565 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":353,"threads_started":1}
I20260808 03:56:16.557664 24119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c963efac28784c00b5ad30c7dfb7ef02: No bootstrap required, opened a new log
I20260808 03:56:16.565189 24119 raft_consensus.cc:348] T 00000000000000000000000000000000 P c963efac28784c00b5ad30c7dfb7ef02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c963efac28784c00b5ad30c7dfb7ef02" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36407 } }
I20260808 03:56:16.565513 24119 raft_consensus.cc:374] T 00000000000000000000000000000000 P c963efac28784c00b5ad30c7dfb7ef02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:16.565589 24119 raft_consensus.cc:729] T 00000000000000000000000000000000 P c963efac28784c00b5ad30c7dfb7ef02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c963efac28784c00b5ad30c7dfb7ef02, State: Initialized, Role: FOLLOWER
I20260808 03:56:16.566109 24119 consensus_queue.cc:260] T 00000000000000000000000000000000 P c963efac28784c00b5ad30c7dfb7ef02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c963efac28784c00b5ad30c7dfb7ef02" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36407 } }
I20260808 03:56:16.566334 24119 raft_consensus.cc:388] T 00000000000000000000000000000000 P c963efac28784c00b5ad30c7dfb7ef02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:56:16.566430 24119 raft_consensus.cc:482] T 00000000000000000000000000000000 P c963efac28784c00b5ad30c7dfb7ef02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:56:16.566552 24119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c963efac28784c00b5ad30c7dfb7ef02 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:16.568988 24119 raft_consensus.cc:504] T 00000000000000000000000000000000 P c963efac28784c00b5ad30c7dfb7ef02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c963efac28784c00b5ad30c7dfb7ef02" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36407 } }
I20260808 03:56:16.569460 24119 leader_election.cc:302] T 00000000000000000000000000000000 P c963efac28784c00b5ad30c7dfb7ef02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c963efac28784c00b5ad30c7dfb7ef02; no voters:
I20260808 03:56:16.569998 24119 leader_election.cc:288] T 00000000000000000000000000000000 P c963efac28784c00b5ad30c7dfb7ef02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:56:16.570151 24126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c963efac28784c00b5ad30c7dfb7ef02 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:16.570916 24126 raft_consensus.cc:686] T 00000000000000000000000000000000 P c963efac28784c00b5ad30c7dfb7ef02 [term 1 LEADER]: Becoming Leader. State: Replica: c963efac28784c00b5ad30c7dfb7ef02, State: Running, Role: LEADER
I20260808 03:56:16.571422 24126 consensus_queue.cc:237] T 00000000000000000000000000000000 P c963efac28784c00b5ad30c7dfb7ef02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c963efac28784c00b5ad30c7dfb7ef02" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36407 } }
I20260808 03:56:16.572067 24119 sys_catalog.cc:564] T 00000000000000000000000000000000 P c963efac28784c00b5ad30c7dfb7ef02 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:16.574476 24132 sys_catalog.cc:455] T 00000000000000000000000000000000 P c963efac28784c00b5ad30c7dfb7ef02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c963efac28784c00b5ad30c7dfb7ef02. Latest consensus state: current_term: 1 leader_uuid: "c963efac28784c00b5ad30c7dfb7ef02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c963efac28784c00b5ad30c7dfb7ef02" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36407 } } }
I20260808 03:56:16.574851 24132 sys_catalog.cc:458] T 00000000000000000000000000000000 P c963efac28784c00b5ad30c7dfb7ef02 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:16.575275 24128 sys_catalog.cc:455] T 00000000000000000000000000000000 P c963efac28784c00b5ad30c7dfb7ef02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c963efac28784c00b5ad30c7dfb7ef02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c963efac28784c00b5ad30c7dfb7ef02" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36407 } } }
I20260808 03:56:16.575562 24128 sys_catalog.cc:458] T 00000000000000000000000000000000 P c963efac28784c00b5ad30c7dfb7ef02 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:16.576936 24141 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:56:16.582612 24141 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:56:16.591939 24141 catalog_manager.cc:1347] Generated new cluster ID: e682349f8ac14a4dadeeea8c4626a237
I20260808 03:56:16.592067 24141 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:56:16.599331 24141 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:56:16.601255 24141 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:56:16.613121 24141 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c963efac28784c00b5ad30c7dfb7ef02: Generated new TSK 0
I20260808 03:56:16.614212 24141 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:56:16.629782 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.129:0
--local_ip_for_outbound_sockets=127.26.146.129
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:36407
--builtin_ntp_servers=127.26.146.148:36833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 03:56:16.973019 24160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:16.973307 24160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:16.973402 24160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:16.980705 24160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:16.980805 24160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:16.980918 24160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.129
I20260808 03:56:16.989998 24160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:36833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.146.129
--webserver_port=0
--tserver_master_addrs=127.26.146.190:36407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.129
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:16.991241 24160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:16.993011 24160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:17.005765 24166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:17.005836 24167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:17.005824 24169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:17.006767 24160 server_base.cc:1034] running on GCE node
I20260808 03:56:17.007351 24160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:17.008311 24160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:17.009552 24160 hybrid_clock.cc:648] HybridClock initialized: now 1786161377009453 us; error 65 us; skew 500 ppm
I20260808 03:56:17.009864 24160 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:56:17.012120 24160 webserver.cc:460] Webserver started at http://127.26.146.129:38781/ using document root <none> and password file <none>
I20260808 03:56:17.012794 24160 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:17.012902 24160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:17.013192 24160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:17.016072 24160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-0/data/instance:
uuid: "1cc0815b35524c69802fefbf550eb664"
format_stamp: "Formatted at 2026-08-08 03:56:17 on dist-test-slave-jlzn"
server_key: "28786d660c0e6f0119fd8fe112505c7c"
server_key_iv: "258f684de6314f67f223e85432e677f7"
server_key_version: "encryptionkey@0"
I20260808 03:56:17.016848 24160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance:
uuid: "1cc0815b35524c69802fefbf550eb664"
format_stamp: "Formatted at 2026-08-08 03:56:17 on dist-test-slave-jlzn"
server_key: "28786d660c0e6f0119fd8fe112505c7c"
server_key_iv: "258f684de6314f67f223e85432e677f7"
server_key_version: "encryptionkey@0"
I20260808 03:56:17.022007 24160 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260808 03:56:17.025578 24182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:17.026870 24160 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 03:56:17.027034 24160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-0/wal
uuid: "1cc0815b35524c69802fefbf550eb664"
format_stamp: "Formatted at 2026-08-08 03:56:17 on dist-test-slave-jlzn"
server_key: "28786d660c0e6f0119fd8fe112505c7c"
server_key_iv: "258f684de6314f67f223e85432e677f7"
server_key_version: "encryptionkey@0"
I20260808 03:56:17.027235 24160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:17.055482 24160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:17.056491 24160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:17.056829 24160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:17.057832 24160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:56:17.059295 24160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:56:17.059394 24160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:17.059502 24160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:56:17.059571 24160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:17.089852 24160 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.129:36715
I20260808 03:56:17.089911 24349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.129:36715 every 8 connection(s)
I20260808 03:56:17.091117 24160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-0/data/info.pb
I20260808 03:56:17.091645 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 24160
I20260808 03:56:17.091825 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-0/wal/instance
I20260808 03:56:17.092374 27210 external_mini_cluster.cc:1428] Setting key 0252474c2624452b33d7a5cb387a7656
I20260808 03:56:17.095288 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.130:0
--local_ip_for_outbound_sockets=127.26.146.130
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:36407
--builtin_ntp_servers=127.26.146.148:36833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 03:56:17.104035 24353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:17.093825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:34699 (local address 127.26.146.190:36407)
0808 03:56:17.094391 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:17.094396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:17.095732 (+ 1336us) server_negotiation.cc:407] Connection header received
0808 03:56:17.097517 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:17.097522 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:17.097595 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:17.097710 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:17.098750 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:17.099685 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:17.101238 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:17.101763 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:17.101884 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:17.103231 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:17.103238 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:17.103242 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:17.103264 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:17.103315 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:17.103329 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:17.103408 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:17.103590 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 03:56:17.103690 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":406,"threads_started":1}
I20260808 03:56:17.104993 24355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:17.094506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36407 (local address 127.26.146.129:34699)
0808 03:56:17.095563 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:17.095619 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:17.097255 (+ 1636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:17.097898 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:17.097905 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:17.097988 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:17.098573 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:17.098600 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:17.099854 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:17.099858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:17.101098 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:17.101111 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:17.101970 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:17.101975 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:17.102197 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:17.102885 (+ 688us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:56:17.102888 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:17.102936 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:56:17.103028 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:17.103416 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:17.103422 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:17.103427 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 03:56:17.103563 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 03:56:17.103879 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":313,"threads_started":1}
I20260808 03:56:17.106487 24350 heartbeater.cc:347] Connected to a master server at 127.26.146.190:36407
I20260808 03:56:17.106774 24350 heartbeater.cc:464] Registering TS with master...
I20260808 03:56:17.107539 24350 heartbeater.cc:511] Master 127.26.146.190:36407 requested a full tablet report, sending...
I20260808 03:56:17.109874 24070 ts_manager.cc:194] Registered new tserver with Master: 1cc0815b35524c69802fefbf550eb664 (127.26.146.129:36715)
I20260808 03:56:17.111430 24070 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.146.129:34699
W20260808 03:56:17.429445 24356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:17.429740 24356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:17.429867 24356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:17.437213 24356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:17.437316 24356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:17.437454 24356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.130
I20260808 03:56:17.446605 24356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:36833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.146.130
--webserver_port=0
--tserver_master_addrs=127.26.146.190:36407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.130
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:17.447960 24356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:17.449757 24356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:17.462442 24364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:17.462442 24370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:17.462955 24362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:17.463287 24356 server_base.cc:1034] running on GCE node
I20260808 03:56:17.463891 24356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:17.464761 24356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:17.465973 24356 hybrid_clock.cc:648] HybridClock initialized: now 1786161377465926 us; error 32 us; skew 500 ppm
I20260808 03:56:17.466253 24356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:56:17.468464 24356 webserver.cc:460] Webserver started at http://127.26.146.130:34893/ using document root <none> and password file <none>
I20260808 03:56:17.469089 24356 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:17.469177 24356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:17.469425 24356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:17.472250 24356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-1/data/instance:
uuid: "7cc241a7026749b4ac8dd2d3de95e348"
format_stamp: "Formatted at 2026-08-08 03:56:17 on dist-test-slave-jlzn"
server_key: "40211cbb76ffa88e91af66a62ca73307"
server_key_iv: "33e76a01f0f5570b481d965056b11c4d"
server_key_version: "encryptionkey@0"
I20260808 03:56:17.473019 24356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance:
uuid: "7cc241a7026749b4ac8dd2d3de95e348"
format_stamp: "Formatted at 2026-08-08 03:56:17 on dist-test-slave-jlzn"
server_key: "40211cbb76ffa88e91af66a62ca73307"
server_key_iv: "33e76a01f0f5570b481d965056b11c4d"
server_key_version: "encryptionkey@0"
I20260808 03:56:17.477910 24356 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260808 03:56:17.481216 24378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:17.482463 24356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 03:56:17.482609 24356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-1/wal
uuid: "7cc241a7026749b4ac8dd2d3de95e348"
format_stamp: "Formatted at 2026-08-08 03:56:17 on dist-test-slave-jlzn"
server_key: "40211cbb76ffa88e91af66a62ca73307"
server_key_iv: "33e76a01f0f5570b481d965056b11c4d"
server_key_version: "encryptionkey@0"
I20260808 03:56:17.482818 24356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:17.503470 24356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:17.504321 24356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:17.504611 24356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:17.505556 24356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:56:17.507054 24356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:56:17.507148 24356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:17.507259 24356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:56:17.507329 24356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:17.537519 24356 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.130:45617
I20260808 03:56:17.537577 24535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.130:45617 every 8 connection(s)
I20260808 03:56:17.538769 24356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-1/data/info.pb
I20260808 03:56:17.547528 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 24356
I20260808 03:56:17.547703 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-1/wal/instance
I20260808 03:56:17.548264 27210 external_mini_cluster.cc:1428] Setting key 6a0b36915cd582a4bb854c8c068d192d
I20260808 03:56:17.551330 24353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:17.541410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.130:58449 (local address 127.26.146.190:36407)
0808 03:56:17.541603 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:17.541610 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:17.543028 (+ 1418us) server_negotiation.cc:407] Connection header received
0808 03:56:17.544841 (+ 1813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:17.544848 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:17.544928 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:17.545017 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:17.546095 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:17.546879 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:17.548561 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:17.549057 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:17.549186 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:17.550576 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:17.550583 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:17.550588 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:17.550609 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:17.550648 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:17.550665 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:17.550761 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:17.550929 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 03:56:17.551029 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 03:56:17.551307 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.146.131:0
--local_ip_for_outbound_sockets=127.26.146.131
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:36407
--builtin_ntp_servers=127.26.146.148:36833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 03:56:17.552289 24539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:17.541952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36407 (local address 127.26.146.130:58449)
0808 03:56:17.542872 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:17.542909 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:17.544572 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:17.545261 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:17.545269 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:17.545353 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:17.545923 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:17.545940 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:17.547032 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:17.547036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:17.548417 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:17.548430 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:17.549287 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:17.549295 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:17.549509 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:17.550276 (+ 767us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:56:17.550279 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:17.550318 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:56:17.550397 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:17.550754 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:17.550760 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:17.550765 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 03:56:17.550893 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 03:56:17.551200 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":286,"threads_started":1}
I20260808 03:56:17.553552 24536 heartbeater.cc:347] Connected to a master server at 127.26.146.190:36407
I20260808 03:56:17.553836 24536 heartbeater.cc:464] Registering TS with master...
I20260808 03:56:17.554555 24536 heartbeater.cc:511] Master 127.26.146.190:36407 requested a full tablet report, sending...
I20260808 03:56:17.555877 24070 ts_manager.cc:194] Registered new tserver with Master: 7cc241a7026749b4ac8dd2d3de95e348 (127.26.146.130:45617)
I20260808 03:56:17.556684 24070 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.146.130:58449
W20260808 03:56:17.889302 24540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:17.889605 24540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:17.889703 24540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:17.897084 24540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:17.897197 24540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:17.897303 24540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.146.131
I20260808 03:56:17.906653 24540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:36833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.146.131
--webserver_port=0
--tserver_master_addrs=127.26.146.190:36407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.146.131
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:17.907971 24540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:17.909865 24540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:17.922828 24552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:17.922899 24547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:17.922902 24548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:17.923960 24540 server_base.cc:1034] running on GCE node
I20260808 03:56:17.924522 24540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:17.925470 24540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:17.926693 24540 hybrid_clock.cc:648] HybridClock initialized: now 1786161377926647 us; error 34 us; skew 500 ppm
I20260808 03:56:17.927008 24540 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:56:17.929239 24540 webserver.cc:460] Webserver started at http://127.26.146.131:36665/ using document root <none> and password file <none>
I20260808 03:56:17.929980 24540 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:17.930086 24540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:17.930402 24540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:17.933368 24540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-2/data/instance:
uuid: "cef602744bc944dba4225e9bfce0a5a5"
format_stamp: "Formatted at 2026-08-08 03:56:17 on dist-test-slave-jlzn"
server_key: "c89c24ecf1504411e5255ade320287a1"
server_key_iv: "94d351845b66e5d9f7bcecc7cdc2b751"
server_key_version: "encryptionkey@0"
I20260808 03:56:17.934108 24540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance:
uuid: "cef602744bc944dba4225e9bfce0a5a5"
format_stamp: "Formatted at 2026-08-08 03:56:17 on dist-test-slave-jlzn"
server_key: "c89c24ecf1504411e5255ade320287a1"
server_key_iv: "94d351845b66e5d9f7bcecc7cdc2b751"
server_key_version: "encryptionkey@0"
I20260808 03:56:17.939134 24540 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 03:56:17.942442 24562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:17.943753 24540 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:56:17.943950 24540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-2/wal
uuid: "cef602744bc944dba4225e9bfce0a5a5"
format_stamp: "Formatted at 2026-08-08 03:56:17 on dist-test-slave-jlzn"
server_key: "c89c24ecf1504411e5255ade320287a1"
server_key_iv: "94d351845b66e5d9f7bcecc7cdc2b751"
server_key_version: "encryptionkey@0"
I20260808 03:56:17.944151 24540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:17.961983 24540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:17.962761 24540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:17.963045 24540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:17.963999 24540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:56:17.965365 24540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:56:17.965466 24540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:17.965592 24540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:56:17.965664 24540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:17.996280 24540 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.131:34755
I20260808 03:56:17.996328 24729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.131:34755 every 8 connection(s)
I20260808 03:56:17.997586 24540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-2/data/info.pb
I20260808 03:56:18.003189 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 24540
I20260808 03:56:18.003367 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-2/wal/instance
I20260808 03:56:18.003950 27210 external_mini_cluster.cc:1428] Setting key e2b60ec6db7a6e3bcf0f70f41828ad8b
I20260808 03:56:18.010334 24353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:17.999983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.131:56061 (local address 127.26.146.190:36407)
0808 03:56:18.000250 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:18.000254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:18.001517 (+ 1263us) server_negotiation.cc:407] Connection header received
0808 03:56:18.003438 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:18.003445 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:18.003534 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:18.003647 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:18.004921 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.005952 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.007757 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.008223 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.008318 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:18.009683 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:18.009688 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:18.009692 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:18.009710 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:18.009741 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:18.009752 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:18.009819 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:18.010016 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 03:56:18.010089 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260808 03:56:18.011348 24733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:18.000491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36407 (local address 127.26.146.131:56061)
0808 03:56:18.001381 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:18.001416 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:18.003167 (+ 1751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:18.003950 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:18.003958 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:18.004073 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:18.004763 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:18.004788 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.006128 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.006134 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:18.007627 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:18.007640 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.008446 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.008452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:18.008722 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:18.009345 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:56:18.009348 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:18.009387 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:56:18.009466 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:18.009841 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:18.009850 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:18.009857 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 03:56:18.010000 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 03:56:18.010292 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":269,"threads_started":1}
I20260808 03:56:18.012696 24731 heartbeater.cc:347] Connected to a master server at 127.26.146.190:36407
I20260808 03:56:18.013007 24731 heartbeater.cc:464] Registering TS with master...
I20260808 03:56:18.013749 24731 heartbeater.cc:511] Master 127.26.146.190:36407 requested a full tablet report, sending...
I20260808 03:56:18.015110 24070 ts_manager.cc:194] Registered new tserver with Master: cef602744bc944dba4225e9bfce0a5a5 (127.26.146.131:34755)
I20260808 03:56:18.015933 24070 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.146.131:56061
I20260808 03:56:18.019311 27210 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:56:18.028723 24353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:18.023207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51892 (local address 127.26.146.190:36407)
0808 03:56:18.023390 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:18.023394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:18.023925 (+ 531us) server_negotiation.cc:407] Connection header received
0808 03:56:18.024075 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:18.024079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:18.024153 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:18.024231 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:18.024686 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.025366 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.026759 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.027161 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.027251 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:18.028230 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:18.028234 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:18.028238 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:18.028256 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:18.028287 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:18.028298 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:18.028365 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:18.028447 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 03:56:18.028516 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 03:56:18.032424 24070 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51892:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:56:18.039165 24070 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:56:18.061739 24751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:18.051205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.129:36715 (local address 127.0.0.1:40916)
0808 03:56:18.052385 (+ 1180us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:18.052433 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:18.052598 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:18.053950 (+ 1352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:18.053958 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:18.053989 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:18.054457 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:18.054472 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.056575 (+ 2103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.056580 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:18.059279 (+ 2699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:18.059292 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.060794 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.060801 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:18.061007 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:18.061070 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 03:56:18.061191 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 03:56:18.061376 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":911,"mutex_wait_us":85,"thread_start_us":768,"threads_started":1}
I20260808 03:56:18.061762 24754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:18.051362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40916 (local address 127.26.146.129:36715)
0808 03:56:18.053352 (+ 1990us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:18.053358 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:18.053381 (+ 23us) server_negotiation.cc:407] Connection header received
0808 03:56:18.053455 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:18.053460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:18.053690 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:18.054838 (+ 1148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:18.055063 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.056426 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.059461 (+ 3035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.060594 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.060766 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:18.060983 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:18.061248 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 03:56:18.061421 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1864,"thread_start_us":1509,"threads_started":1}
I20260808 03:56:18.062945 24758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:18.052017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34396 (local address 127.26.146.130:45617)
0808 03:56:18.052986 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:18.053005 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:18.053538 (+ 533us) server_negotiation.cc:407] Connection header received
0808 03:56:18.053721 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:18.053729 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:18.053942 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:18.054119 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:18.056454 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.057537 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.060282 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.061606 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.061768 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:18.061972 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:18.062452 (+ 480us) server_negotiation.cc:299] Negotiation successful
0808 03:56:18.062637 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":798,"thread_start_us":659,"threads_started":1}
I20260808 03:56:18.063884 24752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:18.051350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:45617 (local address 127.0.0.1:34396)
0808 03:56:18.053416 (+ 2066us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:18.053438 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:18.053562 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:18.055900 (+ 2338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:18.055905 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:18.055932 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:18.056257 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:18.056267 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.057717 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.057723 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:18.060076 (+ 2353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:18.060089 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.062099 (+ 2010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.062104 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:18.062242 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:18.062273 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 03:56:18.063359 (+ 1086us) client_negotiation.cc:240] Negotiation successful
0808 03:56:18.063489 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1933,"mutex_wait_us":1,"thread_start_us":1946,"threads_started":1}
I20260808 03:56:18.064369 24761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:18.053631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:34755 (local address 127.0.0.1:38476)
0808 03:56:18.054654 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:18.054674 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:18.054771 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:18.057563 (+ 2792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:18.057568 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:18.057600 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:18.058200 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:18.058216 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.060207 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.060212 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:18.061914 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:18.061931 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.063604 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.063611 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:18.063793 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:18.063850 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 03:56:18.063931 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 03:56:18.064047 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":904,"mutex_wait_us":76,"thread_start_us":760,"threads_started":1}
I20260808 03:56:18.064489 24762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:18.054122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38476 (local address 127.26.146.131:34755)
0808 03:56:18.056883 (+ 2761us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:18.056891 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:18.056929 (+ 38us) server_negotiation.cc:407] Connection header received
0808 03:56:18.057016 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:18.057024 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:18.057309 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:18.057899 (+ 590us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:18.058451 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.060039 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.062126 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.063418 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.063608 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:18.063795 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:18.063995 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 03:56:18.064142 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2588,"thread_start_us":2388,"threads_started":1}
I20260808 03:56:18.066303 24253 tablet_service.cc:1467] Processing CreateTablet for tablet 499100e63d65486c9222519c7d81da31 (DEFAULT_TABLE table=test-table [id=680f04184ff14d9a80df3a7335f2576b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:56:18.067198 24444 tablet_service.cc:1467] Processing CreateTablet for tablet 499100e63d65486c9222519c7d81da31 (DEFAULT_TABLE table=test-table [id=680f04184ff14d9a80df3a7335f2576b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:56:18.067899 24253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 499100e63d65486c9222519c7d81da31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:18.068533 24444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 499100e63d65486c9222519c7d81da31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:18.068460 24626 tablet_service.cc:1467] Processing CreateTablet for tablet 499100e63d65486c9222519c7d81da31 (DEFAULT_TABLE table=test-table [id=680f04184ff14d9a80df3a7335f2576b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:56:18.069797 24626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 499100e63d65486c9222519c7d81da31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:18.079332 24763 tablet_bootstrap.cc:492] T 499100e63d65486c9222519c7d81da31 P 7cc241a7026749b4ac8dd2d3de95e348: Bootstrap starting.
I20260808 03:56:18.082501 24764 tablet_bootstrap.cc:492] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664: Bootstrap starting.
I20260808 03:56:18.083272 24763 tablet_bootstrap.cc:654] T 499100e63d65486c9222519c7d81da31 P 7cc241a7026749b4ac8dd2d3de95e348: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:18.084233 24765 tablet_bootstrap.cc:492] T 499100e63d65486c9222519c7d81da31 P cef602744bc944dba4225e9bfce0a5a5: Bootstrap starting.
I20260808 03:56:18.084545 24763 log.cc:824] T 499100e63d65486c9222519c7d81da31 P 7cc241a7026749b4ac8dd2d3de95e348: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:18.085948 24764 tablet_bootstrap.cc:654] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:18.087214 24764 log.cc:824] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:18.087266 24765 tablet_bootstrap.cc:654] T 499100e63d65486c9222519c7d81da31 P cef602744bc944dba4225e9bfce0a5a5: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:18.087738 24763 tablet_bootstrap.cc:492] T 499100e63d65486c9222519c7d81da31 P 7cc241a7026749b4ac8dd2d3de95e348: No bootstrap required, opened a new log
I20260808 03:56:18.088071 24763 ts_tablet_manager.cc:1397] T 499100e63d65486c9222519c7d81da31 P 7cc241a7026749b4ac8dd2d3de95e348: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260808 03:56:18.088321 24765 log.cc:824] T 499100e63d65486c9222519c7d81da31 P cef602744bc944dba4225e9bfce0a5a5: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:18.090366 24764 tablet_bootstrap.cc:492] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664: No bootstrap required, opened a new log
I20260808 03:56:18.090611 24764 ts_tablet_manager.cc:1397] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260808 03:56:18.092504 24765 tablet_bootstrap.cc:492] T 499100e63d65486c9222519c7d81da31 P cef602744bc944dba4225e9bfce0a5a5: No bootstrap required, opened a new log
I20260808 03:56:18.092772 24765 ts_tablet_manager.cc:1397] T 499100e63d65486c9222519c7d81da31 P cef602744bc944dba4225e9bfce0a5a5: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 03:56:18.101447 24764 raft_consensus.cc:348] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc0815b35524c69802fefbf550eb664" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 36715 } } peers { permanent_uuid: "7cc241a7026749b4ac8dd2d3de95e348" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45617 } } peers { permanent_uuid: "cef602744bc944dba4225e9bfce0a5a5" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34755 } }
I20260808 03:56:18.101895 24764 raft_consensus.cc:374] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:18.102078 24764 raft_consensus.cc:729] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc0815b35524c69802fefbf550eb664, State: Initialized, Role: FOLLOWER
I20260808 03:56:18.102694 24764 consensus_queue.cc:260] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc0815b35524c69802fefbf550eb664" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 36715 } } peers { permanent_uuid: "7cc241a7026749b4ac8dd2d3de95e348" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45617 } } peers { permanent_uuid: "cef602744bc944dba4225e9bfce0a5a5" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34755 } }
I20260808 03:56:18.102898 24765 raft_consensus.cc:348] T 499100e63d65486c9222519c7d81da31 P cef602744bc944dba4225e9bfce0a5a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc0815b35524c69802fefbf550eb664" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 36715 } } peers { permanent_uuid: "7cc241a7026749b4ac8dd2d3de95e348" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45617 } } peers { permanent_uuid: "cef602744bc944dba4225e9bfce0a5a5" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34755 } }
I20260808 03:56:18.103331 24765 raft_consensus.cc:374] T 499100e63d65486c9222519c7d81da31 P cef602744bc944dba4225e9bfce0a5a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:18.103436 24765 raft_consensus.cc:729] T 499100e63d65486c9222519c7d81da31 P cef602744bc944dba4225e9bfce0a5a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cef602744bc944dba4225e9bfce0a5a5, State: Initialized, Role: FOLLOWER
I20260808 03:56:18.104095 24765 consensus_queue.cc:260] T 499100e63d65486c9222519c7d81da31 P cef602744bc944dba4225e9bfce0a5a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc0815b35524c69802fefbf550eb664" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 36715 } } peers { permanent_uuid: "7cc241a7026749b4ac8dd2d3de95e348" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45617 } } peers { permanent_uuid: "cef602744bc944dba4225e9bfce0a5a5" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34755 } }
I20260808 03:56:18.105263 24731 heartbeater.cc:503] Master 127.26.146.190:36407 was elected leader, sending a full tablet report...
I20260808 03:56:18.105923 24765 ts_tablet_manager.cc:1428] T 499100e63d65486c9222519c7d81da31 P cef602744bc944dba4225e9bfce0a5a5: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260808 03:56:18.105888 24763 raft_consensus.cc:348] T 499100e63d65486c9222519c7d81da31 P 7cc241a7026749b4ac8dd2d3de95e348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc0815b35524c69802fefbf550eb664" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 36715 } } peers { permanent_uuid: "7cc241a7026749b4ac8dd2d3de95e348" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45617 } } peers { permanent_uuid: "cef602744bc944dba4225e9bfce0a5a5" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34755 } }
I20260808 03:56:18.106290 24763 raft_consensus.cc:374] T 499100e63d65486c9222519c7d81da31 P 7cc241a7026749b4ac8dd2d3de95e348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:18.106382 24763 raft_consensus.cc:729] T 499100e63d65486c9222519c7d81da31 P 7cc241a7026749b4ac8dd2d3de95e348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc241a7026749b4ac8dd2d3de95e348, State: Initialized, Role: FOLLOWER
I20260808 03:56:18.107052 24763 consensus_queue.cc:260] T 499100e63d65486c9222519c7d81da31 P 7cc241a7026749b4ac8dd2d3de95e348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc0815b35524c69802fefbf550eb664" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 36715 } } peers { permanent_uuid: "7cc241a7026749b4ac8dd2d3de95e348" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45617 } } peers { permanent_uuid: "cef602744bc944dba4225e9bfce0a5a5" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34755 } }
I20260808 03:56:18.108474 24763 ts_tablet_manager.cc:1428] T 499100e63d65486c9222519c7d81da31 P 7cc241a7026749b4ac8dd2d3de95e348: Time spent starting tablet: real 0.020s user 0.011s sys 0.000s
I20260808 03:56:18.109295 24536 heartbeater.cc:503] Master 127.26.146.190:36407 was elected leader, sending a full tablet report...
I20260808 03:56:18.109302 24350 heartbeater.cc:503] Master 127.26.146.190:36407 was elected leader, sending a full tablet report...
I20260808 03:56:18.109963 24764 ts_tablet_manager.cc:1428] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664: Time spent starting tablet: real 0.019s user 0.012s sys 0.007s
I20260808 03:56:18.148275 24769 raft_consensus.cc:482] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:56:18.148777 24769 raft_consensus.cc:504] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc0815b35524c69802fefbf550eb664" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 36715 } } peers { permanent_uuid: "7cc241a7026749b4ac8dd2d3de95e348" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45617 } } peers { permanent_uuid: "cef602744bc944dba4225e9bfce0a5a5" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34755 } }
I20260808 03:56:18.150700 24769 leader_election.cc:288] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cc241a7026749b4ac8dd2d3de95e348 (127.26.146.130:45617), cef602744bc944dba4225e9bfce0a5a5 (127.26.146.131:34755)
I20260808 03:56:18.156883 24758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:18.151066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:35635 (local address 127.26.146.130:45617)
0808 03:56:18.151271 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:18.151276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:18.151881 (+ 605us) server_negotiation.cc:407] Connection header received
0808 03:56:18.152116 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:18.152121 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:18.152199 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:18.152290 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:18.152856 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.153545 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.155278 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.156073 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.156208 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:18.156278 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:18.156536 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 03:56:18.156651 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 03:56:18.156925 24772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:18.151077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.130:45617 (local address 127.26.146.129:35635)
0808 03:56:18.151750 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:18.151769 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:18.151941 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:18.152332 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:18.152336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:18.152363 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:18.152715 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:18.152726 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.153668 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.153673 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:18.155092 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:18.155104 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.156242 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.156249 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:18.156389 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:18.156412 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 03:56:18.156489 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 03:56:18.156608 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":344,"threads_started":1}
I20260808 03:56:18.157617 24773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:18.151886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.131:34755 (local address 127.26.146.129:57055)
0808 03:56:18.152757 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:18.152770 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:18.152880 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:18.153166 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:18.153170 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:18.153193 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:56:18.153507 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:18.153515 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.154466 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.154471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:18.155837 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:18.155852 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.156962 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.156969 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:18.157139 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:18.157171 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 03:56:18.157254 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 03:56:18.157365 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764,"thread_start_us":530,"threads_started":1}
I20260808 03:56:18.157642 24762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:18.152125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.146.129:57055 (local address 127.26.146.131:34755)
0808 03:56:18.152331 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:18.152338 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:18.152858 (+ 520us) server_negotiation.cc:407] Connection header received
0808 03:56:18.152990 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:18.152996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:18.153065 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:18.153135 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:56:18.153659 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.154338 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.156013 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.156779 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.156918 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:18.156983 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:18.157303 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 03:56:18.157397 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 03:56:18.157976 24473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "499100e63d65486c9222519c7d81da31" candidate_uuid: "1cc0815b35524c69802fefbf550eb664" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc241a7026749b4ac8dd2d3de95e348" is_pre_election: true
I20260808 03:56:18.158540 24473 raft_consensus.cc:2454] T 499100e63d65486c9222519c7d81da31 P 7cc241a7026749b4ac8dd2d3de95e348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc0815b35524c69802fefbf550eb664 in term 0.
I20260808 03:56:18.158686 24655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "499100e63d65486c9222519c7d81da31" candidate_uuid: "1cc0815b35524c69802fefbf550eb664" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cef602744bc944dba4225e9bfce0a5a5" is_pre_election: true
I20260808 03:56:18.159168 24655 raft_consensus.cc:2454] T 499100e63d65486c9222519c7d81da31 P cef602744bc944dba4225e9bfce0a5a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc0815b35524c69802fefbf550eb664 in term 0.
I20260808 03:56:18.159175 24188 leader_election.cc:302] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc0815b35524c69802fefbf550eb664, 7cc241a7026749b4ac8dd2d3de95e348; no voters:
I20260808 03:56:18.159530 24769 raft_consensus.cc:2781] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:56:18.159662 24769 raft_consensus.cc:482] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:56:18.159766 24769 raft_consensus.cc:3037] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:18.161931 24769 raft_consensus.cc:504] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc0815b35524c69802fefbf550eb664" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 36715 } } peers { permanent_uuid: "7cc241a7026749b4ac8dd2d3de95e348" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45617 } } peers { permanent_uuid: "cef602744bc944dba4225e9bfce0a5a5" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34755 } }
I20260808 03:56:18.162462 24769 leader_election.cc:288] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664 [CANDIDATE]: Term 1 election: Requested vote from peers 7cc241a7026749b4ac8dd2d3de95e348 (127.26.146.130:45617), cef602744bc944dba4225e9bfce0a5a5 (127.26.146.131:34755)
I20260808 03:56:18.162869 24473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "499100e63d65486c9222519c7d81da31" candidate_uuid: "1cc0815b35524c69802fefbf550eb664" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc241a7026749b4ac8dd2d3de95e348"
I20260808 03:56:18.162921 24655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "499100e63d65486c9222519c7d81da31" candidate_uuid: "1cc0815b35524c69802fefbf550eb664" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cef602744bc944dba4225e9bfce0a5a5"
I20260808 03:56:18.163030 24473 raft_consensus.cc:3037] T 499100e63d65486c9222519c7d81da31 P 7cc241a7026749b4ac8dd2d3de95e348 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:18.163098 24655 raft_consensus.cc:3037] T 499100e63d65486c9222519c7d81da31 P cef602744bc944dba4225e9bfce0a5a5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:18.165266 24655 raft_consensus.cc:2454] T 499100e63d65486c9222519c7d81da31 P cef602744bc944dba4225e9bfce0a5a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc0815b35524c69802fefbf550eb664 in term 1.
I20260808 03:56:18.165333 24473 raft_consensus.cc:2454] T 499100e63d65486c9222519c7d81da31 P 7cc241a7026749b4ac8dd2d3de95e348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc0815b35524c69802fefbf550eb664 in term 1.
I20260808 03:56:18.165618 24188 leader_election.cc:302] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc0815b35524c69802fefbf550eb664, cef602744bc944dba4225e9bfce0a5a5; no voters:
I20260808 03:56:18.165855 24769 raft_consensus.cc:2781] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:18.166290 24769 raft_consensus.cc:686] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664 [term 1 LEADER]: Becoming Leader. State: Replica: 1cc0815b35524c69802fefbf550eb664, State: Running, Role: LEADER
I20260808 03:56:18.166751 24769 consensus_queue.cc:237] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc0815b35524c69802fefbf550eb664" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 36715 } } peers { permanent_uuid: "7cc241a7026749b4ac8dd2d3de95e348" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45617 } } peers { permanent_uuid: "cef602744bc944dba4225e9bfce0a5a5" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34755 } }
I20260808 03:56:18.171015 24069 catalog_manager.cc:5495] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cc0815b35524c69802fefbf550eb664 (127.26.146.129). New cstate: current_term: 1 leader_uuid: "1cc0815b35524c69802fefbf550eb664" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc0815b35524c69802fefbf550eb664" member_type: VOTER last_known_addr { host: "127.26.146.129" port: 36715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc241a7026749b4ac8dd2d3de95e348" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cef602744bc944dba4225e9bfce0a5a5" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34755 } health_report { overall_health: UNKNOWN } } }
I20260808 03:56:18.215682 24754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:18.210515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40922 (local address 127.26.146.129:36715)
0808 03:56:18.210715 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:18.210720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:18.210739 (+ 19us) server_negotiation.cc:407] Connection header received
0808 03:56:18.210879 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:18.210883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:18.210970 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:18.211117 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:56:18.211432 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.212204 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.213573 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.213983 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.214084 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:18.214597 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:56:18.215120 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:56:18.215233 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:18.215346 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 03:56:18.215410 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 03:56:18.223866 24473 raft_consensus.cc:1261] T 499100e63d65486c9222519c7d81da31 P 7cc241a7026749b4ac8dd2d3de95e348 [term 1 FOLLOWER]: Refusing update from remote peer 1cc0815b35524c69802fefbf550eb664: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:56:18.223876 24655 raft_consensus.cc:1261] T 499100e63d65486c9222519c7d81da31 P cef602744bc944dba4225e9bfce0a5a5 [term 1 FOLLOWER]: Refusing update from remote peer 1cc0815b35524c69802fefbf550eb664: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:56:18.224781 24778 consensus_queue.cc:1035] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc241a7026749b4ac8dd2d3de95e348" member_type: VOTER last_known_addr { host: "127.26.146.130" port: 45617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:56:18.225117 24769 consensus_queue.cc:1035] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cef602744bc944dba4225e9bfce0a5a5" member_type: VOTER last_known_addr { host: "127.26.146.131" port: 34755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:56:18.236115 24781 mvcc.cc:205] Tried to move back new op lower bound from 7316117005195538432 to 7316117004974481408. Current Snapshot: MvccSnapshot[applied={T|T < 7316117005195538432}]
I20260808 03:56:18.252092 24069 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:56:18.252405 24069 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:56:18.255982 24069 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c963efac28784c00b5ad30c7dfb7ef02: Sending DeleteTablet for 3 replicas of tablet 499100e63d65486c9222519c7d81da31
I20260808 03:56:18.256983 24253 tablet_service.cc:1514] Processing DeleteTablet for tablet 499100e63d65486c9222519c7d81da31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:56:18 UTC) from {username='slave'} at 127.0.0.1:40916
I20260808 03:56:18.257185 24444 tablet_service.cc:1514] Processing DeleteTablet for tablet 499100e63d65486c9222519c7d81da31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:56:18 UTC) from {username='slave'} at 127.0.0.1:34396
I20260808 03:56:18.257596 24626 tablet_service.cc:1514] Processing DeleteTablet for tablet 499100e63d65486c9222519c7d81da31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:56:18 UTC) from {username='slave'} at 127.0.0.1:38476
I20260808 03:56:18.258384 24796 tablet_replica.cc:330] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664: stopping tablet replica
I20260808 03:56:18.258481 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 24160
I20260808 03:56:18.258503 24798 tablet_replica.cc:330] T 499100e63d65486c9222519c7d81da31 P 7cc241a7026749b4ac8dd2d3de95e348: stopping tablet replica
I20260808 03:56:18.258910 24798 raft_consensus.cc:2229] T 499100e63d65486c9222519c7d81da31 P 7cc241a7026749b4ac8dd2d3de95e348 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:56:18.258915 24796 raft_consensus.cc:2229] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664 [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:56:18.259423 24798 raft_consensus.cc:2258] T 499100e63d65486c9222519c7d81da31 P 7cc241a7026749b4ac8dd2d3de95e348 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:56:18.259876 24796 raft_consensus.cc:2258] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:56:18.260157 24799 tablet_replica.cc:330] T 499100e63d65486c9222519c7d81da31 P cef602744bc944dba4225e9bfce0a5a5: stopping tablet replica
I20260808 03:56:18.260514 24799 raft_consensus.cc:2229] T 499100e63d65486c9222519c7d81da31 P cef602744bc944dba4225e9bfce0a5a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:56:18.260958 24799 raft_consensus.cc:2258] T 499100e63d65486c9222519c7d81da31 P cef602744bc944dba4225e9bfce0a5a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:56:18.261968 24798 ts_tablet_manager.cc:1905] T 499100e63d65486c9222519c7d81da31 P 7cc241a7026749b4ac8dd2d3de95e348: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:56:18.263247 24796 ts_tablet_manager.cc:1905] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:56:18.268643 24798 ts_tablet_manager.cc:1918] T 499100e63d65486c9222519c7d81da31 P 7cc241a7026749b4ac8dd2d3de95e348: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:56:18.268808 24798 log.cc:1196] T 499100e63d65486c9222519c7d81da31 P 7cc241a7026749b4ac8dd2d3de95e348: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-1/wal/wals/499100e63d65486c9222519c7d81da31
I20260808 03:56:18.269342 24798 ts_tablet_manager.cc:1939] T 499100e63d65486c9222519c7d81da31 P 7cc241a7026749b4ac8dd2d3de95e348: Deleting consensus metadata
I20260808 03:56:18.270275 24796 ts_tablet_manager.cc:1918] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:56:18.270462 24796 log.cc:1196] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-0/wal/wals/499100e63d65486c9222519c7d81da31
I20260808 03:56:18.270993 24796 ts_tablet_manager.cc:1939] T 499100e63d65486c9222519c7d81da31 P 1cc0815b35524c69802fefbf550eb664: Deleting consensus metadata
I20260808 03:56:18.271243 24053 catalog_manager.cc:4841] TS 7cc241a7026749b4ac8dd2d3de95e348 (127.26.146.130:45617): tablet 499100e63d65486c9222519c7d81da31 (table test-table [id=680f04184ff14d9a80df3a7335f2576b]) successfully deleted
I20260808 03:56:18.272761 24799 ts_tablet_manager.cc:1905] T 499100e63d65486c9222519c7d81da31 P cef602744bc944dba4225e9bfce0a5a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:56:18.272754 24052 catalog_manager.cc:4841] TS 1cc0815b35524c69802fefbf550eb664 (127.26.146.129:36715): tablet 499100e63d65486c9222519c7d81da31 (table test-table [id=680f04184ff14d9a80df3a7335f2576b]) successfully deleted
I20260808 03:56:18.279752 24799 ts_tablet_manager.cc:1918] T 499100e63d65486c9222519c7d81da31 P cef602744bc944dba4225e9bfce0a5a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:56:18.279964 24799 log.cc:1196] T 499100e63d65486c9222519c7d81da31 P cef602744bc944dba4225e9bfce0a5a5: Deleting WAL directory at /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786161058098901-27210-0/minicluster-data/ts-2/wal/wals/499100e63d65486c9222519c7d81da31
I20260808 03:56:18.280479 24799 ts_tablet_manager.cc:1939] T 499100e63d65486c9222519c7d81da31 P cef602744bc944dba4225e9bfce0a5a5: Deleting consensus metadata
I20260808 03:56:18.282230 24053 catalog_manager.cc:4841] TS cef602744bc944dba4225e9bfce0a5a5 (127.26.146.131:34755): tablet 499100e63d65486c9222519c7d81da31 (table test-table [id=680f04184ff14d9a80df3a7335f2576b]) successfully deleted
I20260808 03:56:18.282696 24754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:18.260062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40934 (local address 127.26.146.129:36715)
0808 03:56:18.260234 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:18.260240 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:18.272209 (+ 11969us) server_negotiation.cc:407] Connection header received
0808 03:56:18.272495 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:18.272500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:18.272645 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:18.272717 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:18.273333 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.274184 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.275589 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.276054 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.276175 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:18.280964 (+ 4789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:18.280973 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:18.280988 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:18.281615 (+ 627us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:18.281667 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:18.281688 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:18.281780 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:18.281934 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 03:56:18.282045 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 03:56:18.283337 24343 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:18.378018 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 24160
I20260808 03:56:18.402235 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 24356
I20260808 03:56:18.409205 24758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:18.403080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34406 (local address 127.26.146.130:45617)
0808 03:56:18.403272 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:18.403278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:18.403299 (+ 21us) server_negotiation.cc:407] Connection header received
0808 03:56:18.403406 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:18.403411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:18.403520 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:18.403609 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:18.404090 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.404950 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.406222 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.406645 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.406761 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:18.407781 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:18.407790 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:18.407825 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:18.408437 (+ 612us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:18.408500 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:18.408523 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:18.408625 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:18.408773 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 03:56:18.408883 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 03:56:18.409927 24529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:18.497184 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 24356
I20260808 03:56:18.519207 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 24540
I20260808 03:56:18.525899 24762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:18.519947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38486 (local address 127.26.146.131:34755)
0808 03:56:18.520121 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:18.520152 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:18.520174 (+ 22us) server_negotiation.cc:407] Connection header received
0808 03:56:18.520368 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:18.520373 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:18.520463 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:18.520564 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:18.520998 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.521741 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.522977 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:18.523381 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:18.523488 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:18.524575 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:18.524584 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:18.524601 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:18.525184 (+ 583us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:18.525234 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:18.525255 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:18.525365 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:18.525488 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 03:56:18.525599 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 03:56:18.526525 24722 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:18.613801 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 24540
I20260808 03:56:18.637418 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 24027
I20260808 03:56:18.638538 24114 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:18.748570 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 24027
2026-08-08T03:56:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2726 ms)
[----------] 6 tests from EncryptionPolicyTest (22313 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:56:18 dist-test-slave-jlzn krb5kdc[24811](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:56:18 dist-test-slave-jlzn krb5kdc[24811](info): set up 2 sockets
Aug 08 03:56:18 dist-test-slave-jlzn krb5kdc[24811](info): commencing operation
krb5kdc: starting...
W20260808 03:56:19.980576 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.010s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:56:20 dist-test-slave-jlzn krb5kdc[24811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:56:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:56:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 03:56:20.273733 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37109
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:20.619931 24830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:20.620226 24830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:20.620364 24830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:20.627416 24830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:20.627542 24830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:20.627624 24830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:20.627713 24830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:20.627774 24830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:20.637444 24830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37109
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:20.638718 24830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:20.640009 24830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:20.651376 24845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:20.651443 24847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:20.651912 24844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:20.652258 24830 server_base.cc:1034] running on GCE node
I20260808 03:56:20.652999 24830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:20.654392 24830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:20.655604 24830 hybrid_clock.cc:648] HybridClock initialized: now 1786161380655554 us; error 46 us; skew 500 ppm
I20260808 03:56:20.655984 24830 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:20 dist-test-slave-jlzn krb5kdc[24811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161380, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:20.659739 24830 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 03:56:20.661233 24830 webserver.cc:460] Webserver started at http://127.0.0.1:45495/ using document root <none> and password file <none>
I20260808 03:56:20.661866 24830 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:20.661967 24830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:20.662220 24830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:20.665187 24830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "27ad59fdd13947c1b81282bbf2d8be55"
format_stamp: "Formatted at 2026-08-08 03:56:20 on dist-test-slave-jlzn"
server_key: "f4607aa3d7562e664cc2835c2186b7dc"
server_key_iv: "50de324207a81af2dcba6a62a090e217"
server_key_version: "encryptionkey@0"
I20260808 03:56:20.665910 24830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "27ad59fdd13947c1b81282bbf2d8be55"
format_stamp: "Formatted at 2026-08-08 03:56:20 on dist-test-slave-jlzn"
server_key: "f4607aa3d7562e664cc2835c2186b7dc"
server_key_iv: "50de324207a81af2dcba6a62a090e217"
server_key_version: "encryptionkey@0"
I20260808 03:56:20.671090 24830 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 03:56:20.674420 24857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:20.675658 24830 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:56:20.675836 24830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "27ad59fdd13947c1b81282bbf2d8be55"
format_stamp: "Formatted at 2026-08-08 03:56:20 on dist-test-slave-jlzn"
server_key: "f4607aa3d7562e664cc2835c2186b7dc"
server_key_iv: "50de324207a81af2dcba6a62a090e217"
server_key_version: "encryptionkey@0"
I20260808 03:56:20.676010 24830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:20.705109 24830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:20.709352 24830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:20.709631 24830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:20.727485 24830 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37109
I20260808 03:56:20.727583 24928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37109 every 8 connection(s)
I20260808 03:56:20.729013 24830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:56:20.732506 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 24830
I20260808 03:56:20.732713 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:56:20.732910 24929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:20.733398 27210 external_mini_cluster.cc:1428] Setting key de4a5089fd7c044c66e8a9760bac9df6
I20260808 03:56:20.747052 24929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27ad59fdd13947c1b81282bbf2d8be55: Bootstrap starting.
Aug 08 03:56:20 dist-test-slave-jlzn krb5kdc[24811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 03:56:20.750703 24929 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27ad59fdd13947c1b81282bbf2d8be55: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:20.752152 24929 log.cc:824] T 00000000000000000000000000000000 P 27ad59fdd13947c1b81282bbf2d8be55: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:20.755628 24929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27ad59fdd13947c1b81282bbf2d8be55: No bootstrap required, opened a new log
I20260808 03:56:20.757457 24934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:20.735333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50203 (local address 127.0.0.1:37109)
0808 03:56:20.736188 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:20.736203 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:20.736259 (+ 56us) server_negotiation.cc:407] Connection header received
0808 03:56:20.737989 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:20.738016 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:20.738523 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:20.738965 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:20.740305 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:20.742226 (+ 1921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:20.743509 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:20.744614 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:20.744810 (+ 196us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:20.748795 (+ 3985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:20.748805 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:20.748825 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:20.748880 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:20.752648 (+ 3768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:20.753714 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:20.753722 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:20.753730 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:20.753946 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:20.754353 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:20.754357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:20.754360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:20.755080 (+ 720us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:20.755395 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:20.755770 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 03:56:20.756126 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":361,"threads_started":1}
I20260808 03:56:20.761695 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:56:20.764911 24929 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27ad59fdd13947c1b81282bbf2d8be55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ad59fdd13947c1b81282bbf2d8be55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37109 } }
I20260808 03:56:20.765321 24929 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27ad59fdd13947c1b81282bbf2d8be55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:20.765425 24929 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27ad59fdd13947c1b81282bbf2d8be55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27ad59fdd13947c1b81282bbf2d8be55, State: Initialized, Role: FOLLOWER
I20260808 03:56:20.765949 24929 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27ad59fdd13947c1b81282bbf2d8be55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ad59fdd13947c1b81282bbf2d8be55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37109 } }
I20260808 03:56:20.766172 24929 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27ad59fdd13947c1b81282bbf2d8be55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:56:20.766300 24929 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27ad59fdd13947c1b81282bbf2d8be55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:56:20.766541 24929 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27ad59fdd13947c1b81282bbf2d8be55 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:20.769716 24929 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27ad59fdd13947c1b81282bbf2d8be55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ad59fdd13947c1b81282bbf2d8be55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37109 } }
I20260808 03:56:20.770205 24929 leader_election.cc:302] T 00000000000000000000000000000000 P 27ad59fdd13947c1b81282bbf2d8be55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27ad59fdd13947c1b81282bbf2d8be55; no voters:
I20260808 03:56:20.770870 24929 leader_election.cc:288] T 00000000000000000000000000000000 P 27ad59fdd13947c1b81282bbf2d8be55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:56:20.770968 24947 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27ad59fdd13947c1b81282bbf2d8be55 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:20.772953 24929 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27ad59fdd13947c1b81282bbf2d8be55 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:20.778343 24947 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27ad59fdd13947c1b81282bbf2d8be55 [term 1 LEADER]: Becoming Leader. State: Replica: 27ad59fdd13947c1b81282bbf2d8be55, State: Running, Role: LEADER
I20260808 03:56:20.779107 24947 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27ad59fdd13947c1b81282bbf2d8be55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ad59fdd13947c1b81282bbf2d8be55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37109 } }
I20260808 03:56:20.782902 24950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27ad59fdd13947c1b81282bbf2d8be55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27ad59fdd13947c1b81282bbf2d8be55. Latest consensus state: current_term: 1 leader_uuid: "27ad59fdd13947c1b81282bbf2d8be55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ad59fdd13947c1b81282bbf2d8be55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37109 } } }
I20260808 03:56:20.783313 24950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27ad59fdd13947c1b81282bbf2d8be55 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:20.784642 24934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:20.766613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55323 (local address 127.0.0.1:37109)
0808 03:56:20.766799 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:20.766809 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:20.767235 (+ 426us) server_negotiation.cc:407] Connection header received
0808 03:56:20.767400 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:20.767404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:20.767474 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:20.767629 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:20.769392 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:20.770218 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:20.772056 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:20.772807 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:20.772939 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:20.776121 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:20.776128 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:20.776133 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:20.776174 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:20.780318 (+ 4144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:20.781156 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:20.781163 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:20.781166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:20.781324 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:20.781999 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:20.782004 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:20.782007 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:20.782446 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:20.782752 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:20.784043 (+ 1291us) server_negotiation.cc:299] Negotiation successful
0808 03:56:20.784242 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 03:56:20.794266 24958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:56:20.794167 24948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27ad59fdd13947c1b81282bbf2d8be55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27ad59fdd13947c1b81282bbf2d8be55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ad59fdd13947c1b81282bbf2d8be55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37109 } } }
I20260808 03:56:20.794504 24948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27ad59fdd13947c1b81282bbf2d8be55 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:20.799672 24958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:56:20.808766 24958 catalog_manager.cc:1347] Generated new cluster ID: ae85488d277a4077a71656a7907c9ad9
I20260808 03:56:20.808879 24958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:56:20.841004 24958 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:56:20.842293 24958 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:56:20.852787 24958 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27ad59fdd13947c1b81282bbf2d8be55: Generated new TSK 0
I20260808 03:56:20.853571 24958 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:56:20.866911 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 24830
I20260808 03:56:20.869068 24923 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:20.964478 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 24830
W20260808 03:56:20.987237 24809 connection.cc:535] client connection to 127.0.0.1:37109 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37109): Connection reset by peer (error 104)
2026-08-08T03:56:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2210 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:56:21 dist-test-slave-jlzn krb5kdc[24972](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:56:21 dist-test-slave-jlzn krb5kdc[24972](info): set up 2 sockets
Aug 08 03:56:21 dist-test-slave-jlzn krb5kdc[24972](info): commencing operation
krb5kdc: starting...
W20260808 03:56:22.161217 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:56:22 dist-test-slave-jlzn krb5kdc[24972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:56:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:56:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 03:56:22.464485 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33641
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:22.805263 24987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:22.805600 24987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:22.805750 24987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:22.813154 24987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:22.813265 24987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:22.813341 24987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:22.813421 24987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:22.813488 24987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:22.823495 24987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33641
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:22.824867 24987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:22.826198 24987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:22.837607 24994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:22.837733 24998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:22.837644 24995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:22.838620 24987 server_base.cc:1034] running on GCE node
I20260808 03:56:22.839361 24987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:22.840781 24987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:22.842025 24987 hybrid_clock.cc:648] HybridClock initialized: now 1786161382841972 us; error 30 us; skew 500 ppm
I20260808 03:56:22.842383 24987 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:22 dist-test-slave-jlzn krb5kdc[24972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161382, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:22.845609 24987 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 03:56:22.847081 24987 webserver.cc:460] Webserver started at http://127.0.0.1:36913/ using document root <none> and password file <none>
I20260808 03:56:22.847731 24987 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:22.847870 24987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:22.848179 24987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:22.851107 24987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "e9bce50db7704cd59014d16d724235f9"
format_stamp: "Formatted at 2026-08-08 03:56:22 on dist-test-slave-jlzn"
server_key: "4bbcd31b1a1b22d1d948093a7d8357cc"
server_key_iv: "ce4f26a4cb15407a2bab81f5197f8853"
server_key_version: "encryptionkey@0"
I20260808 03:56:22.851928 24987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "e9bce50db7704cd59014d16d724235f9"
format_stamp: "Formatted at 2026-08-08 03:56:22 on dist-test-slave-jlzn"
server_key: "4bbcd31b1a1b22d1d948093a7d8357cc"
server_key_iv: "ce4f26a4cb15407a2bab81f5197f8853"
server_key_version: "encryptionkey@0"
I20260808 03:56:22.856863 24987 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 03:56:22.860321 25006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:22.861526 24987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 03:56:22.861706 24987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "e9bce50db7704cd59014d16d724235f9"
format_stamp: "Formatted at 2026-08-08 03:56:22 on dist-test-slave-jlzn"
server_key: "4bbcd31b1a1b22d1d948093a7d8357cc"
server_key_iv: "ce4f26a4cb15407a2bab81f5197f8853"
server_key_version: "encryptionkey@0"
I20260808 03:56:22.861891 24987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:22.879928 24987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:22.883647 24987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:22.883965 24987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:22.901291 24987 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33641
I20260808 03:56:22.901330 25085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33641 every 8 connection(s)
I20260808 03:56:22.902647 24987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:56:22.903774 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 24987
I20260808 03:56:22.904023 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:56:22.904608 27210 external_mini_cluster.cc:1428] Setting key 6196f931303108fbf362231057a97de6
I20260808 03:56:22.907266 25088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:22.917904 25092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:22.906445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46413 (local address 127.0.0.1:33641)
0808 03:56:22.907321 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:22.907333 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:22.907377 (+ 44us) server_negotiation.cc:407] Connection header received
0808 03:56:22.909032 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:22.909052 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:22.909396 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:22.909708 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:22.910260 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:22.911599 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:22.913461 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:22.914124 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:22.914250 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:22.915572 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:22.915579 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:22.915593 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:22.916187 (+ 594us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:22.916258 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:22.916288 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:22.916409 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:22.916777 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 03:56:22.916962 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":365,"threads_started":1}
I20260808 03:56:22.919174 25088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9bce50db7704cd59014d16d724235f9: Bootstrap starting.
I20260808 03:56:22.921643 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:56:22.922401 25088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9bce50db7704cd59014d16d724235f9: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:22.923588 25088 log.cc:824] T 00000000000000000000000000000000 P e9bce50db7704cd59014d16d724235f9: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:22.926901 25088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9bce50db7704cd59014d16d724235f9: No bootstrap required, opened a new log
I20260808 03:56:22.933579 25092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:22.926669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50059 (local address 127.0.0.1:33641)
0808 03:56:22.926844 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:22.926850 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:22.927384 (+ 534us) server_negotiation.cc:407] Connection header received
0808 03:56:22.927542 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:22.927548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:22.927627 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:22.927744 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:22.928250 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:22.929416 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:22.930798 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:22.931506 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:22.931637 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:22.932788 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:22.932794 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:22.932803 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:22.932824 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:22.932887 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:22.932913 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:22.933055 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:22.933172 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 03:56:22.933239 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 03:56:22.936510 25088 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9bce50db7704cd59014d16d724235f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9bce50db7704cd59014d16d724235f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33641 } }
I20260808 03:56:22.936865 25088 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9bce50db7704cd59014d16d724235f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:22.936939 25088 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9bce50db7704cd59014d16d724235f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bce50db7704cd59014d16d724235f9, State: Initialized, Role: FOLLOWER
I20260808 03:56:22.937469 25088 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9bce50db7704cd59014d16d724235f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9bce50db7704cd59014d16d724235f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33641 } }
I20260808 03:56:22.937687 25088 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9bce50db7704cd59014d16d724235f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:56:22.937788 25088 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9bce50db7704cd59014d16d724235f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:56:22.937915 25088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9bce50db7704cd59014d16d724235f9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:22.940512 25088 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9bce50db7704cd59014d16d724235f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9bce50db7704cd59014d16d724235f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33641 } }
I20260808 03:56:22.940958 25088 leader_election.cc:302] T 00000000000000000000000000000000 P e9bce50db7704cd59014d16d724235f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9bce50db7704cd59014d16d724235f9; no voters:
I20260808 03:56:22.941475 25088 leader_election.cc:288] T 00000000000000000000000000000000 P e9bce50db7704cd59014d16d724235f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:56:22.941601 25101 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9bce50db7704cd59014d16d724235f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:22.942303 25101 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9bce50db7704cd59014d16d724235f9 [term 1 LEADER]: Becoming Leader. State: Replica: e9bce50db7704cd59014d16d724235f9, State: Running, Role: LEADER
I20260808 03:56:22.942806 25101 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9bce50db7704cd59014d16d724235f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9bce50db7704cd59014d16d724235f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33641 } }
I20260808 03:56:22.943521 25088 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9bce50db7704cd59014d16d724235f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:22.945875 25103 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9bce50db7704cd59014d16d724235f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9bce50db7704cd59014d16d724235f9. Latest consensus state: current_term: 1 leader_uuid: "e9bce50db7704cd59014d16d724235f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9bce50db7704cd59014d16d724235f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33641 } } }
I20260808 03:56:22.946274 25103 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9bce50db7704cd59014d16d724235f9 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:22.951236 25102 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9bce50db7704cd59014d16d724235f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9bce50db7704cd59014d16d724235f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9bce50db7704cd59014d16d724235f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33641 } } }
I20260808 03:56:22.951623 25102 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9bce50db7704cd59014d16d724235f9 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:22.952354 25107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:56:22.957221 25107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:56:22.966389 25107 catalog_manager.cc:1347] Generated new cluster ID: 8718da0c98e54d129a8f5e9b324e051a
I20260808 03:56:22.966511 25107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:56:22.992257 25107 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:56:22.993607 25107 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:56:22.998534 25107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9bce50db7704cd59014d16d724235f9: Generated new TSK 0
I20260808 03:56:22.999293 25107 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:56:23.008221 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 24987
W20260808 03:56:23.009393 25079 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46413
W20260808 03:56:23.010195 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 24987: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:56:23.010363 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 24987
W20260808 03:56:23.032975 24969 connection.cc:535] client connection to 127.0.0.1:33641 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33641): Connection reset by peer (error 104)
2026-08-08T03:56:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2045 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:56:23 dist-test-slave-jlzn krb5kdc[25127](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:56:23 dist-test-slave-jlzn krb5kdc[25127](info): set up 2 sockets
Aug 08 03:56:23 dist-test-slave-jlzn krb5kdc[25127](info): commencing operation
krb5kdc: starting...
W20260808 03:56:24.220858 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:56:24 dist-test-slave-jlzn krb5kdc[25127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:56:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:56:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 03:56:24.515329 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43493
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:24.853346 25143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:24.853624 25143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:24.853717 25143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:24.860772 25143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:24.860883 25143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:24.860944 25143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:24.861004 25143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:24.861055 25143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:24.873085 25143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43493
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:24.874806 25143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:24.876573 25143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:24.888772 25150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:24.888840 25151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:24.888859 25153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:24.889918 25143 server_base.cc:1034] running on GCE node
I20260808 03:56:24.890700 25143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:24.892148 25143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:24.893358 25143 hybrid_clock.cc:648] HybridClock initialized: now 1786161384893313 us; error 30 us; skew 500 ppm
I20260808 03:56:24.893679 25143 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:24 dist-test-slave-jlzn krb5kdc[25127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161384, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:24.900660 25143 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 03:56:24.902148 25143 webserver.cc:460] Webserver started at http://127.0.0.1:34625/ using document root <none> and password file <none>
I20260808 03:56:24.902796 25143 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:24.902908 25143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:24.903211 25143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:24.906221 25143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "45cd1e66bf834090ba8646a0e79072d9"
format_stamp: "Formatted at 2026-08-08 03:56:24 on dist-test-slave-jlzn"
server_key: "d965f501c78f92e825e5e7c2eff7ace3"
server_key_iv: "dfb13ecee34d3b29d5a9af1d5adc1f70"
server_key_version: "encryptionkey@0"
I20260808 03:56:24.906975 25143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "45cd1e66bf834090ba8646a0e79072d9"
format_stamp: "Formatted at 2026-08-08 03:56:24 on dist-test-slave-jlzn"
server_key: "d965f501c78f92e825e5e7c2eff7ace3"
server_key_iv: "dfb13ecee34d3b29d5a9af1d5adc1f70"
server_key_version: "encryptionkey@0"
I20260808 03:56:24.912117 25143 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260808 03:56:24.915385 25164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:24.916702 25143 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:56:24.916867 25143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "45cd1e66bf834090ba8646a0e79072d9"
format_stamp: "Formatted at 2026-08-08 03:56:24 on dist-test-slave-jlzn"
server_key: "d965f501c78f92e825e5e7c2eff7ace3"
server_key_iv: "dfb13ecee34d3b29d5a9af1d5adc1f70"
server_key_version: "encryptionkey@0"
I20260808 03:56:24.917059 25143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:24.922737 25143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:24.926880 25143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:24.927161 25143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:24.944540 25143 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43493
I20260808 03:56:24.944628 25244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43493 every 8 connection(s)
I20260808 03:56:24.945900 25143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:56:24.949611 25246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:24.954640 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 25143
I20260808 03:56:24.954807 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:56:24.955214 27210 external_mini_cluster.cc:1428] Setting key f34fdf2beda5b8c20fcfcde8c5dd86c9
I20260808 03:56:24.960409 25246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45cd1e66bf834090ba8646a0e79072d9: Bootstrap starting.
I20260808 03:56:24.962616 25249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:24.956969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51469 (local address 127.0.0.1:43493)
0808 03:56:24.957752 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:24.957762 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:24.957809 (+ 47us) server_negotiation.cc:407] Connection header received
0808 03:56:24.959465 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:24.959487 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:24.959792 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:24.960138 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:24.961171 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:24.961177 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:24.961189 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:24.961203 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:24.961262 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:24.961281 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:24.961440 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:24.961562 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 03:56:24.961717 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":333,"threads_started":1}
I20260808 03:56:24.963693 25246 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45cd1e66bf834090ba8646a0e79072d9: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:24.964962 25246 log.cc:824] T 00000000000000000000000000000000 P 45cd1e66bf834090ba8646a0e79072d9: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:24.966437 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:56:24.968241 25246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45cd1e66bf834090ba8646a0e79072d9: No bootstrap required, opened a new log
I20260808 03:56:24.973870 25249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:24.971565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43893 (local address 127.0.0.1:43493)
0808 03:56:24.971744 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:24.971754 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:24.972153 (+ 399us) server_negotiation.cc:407] Connection header received
0808 03:56:24.972286 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:24.972291 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:24.972350 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:24.972452 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:24.973248 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:24.973254 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:24.973259 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:24.973275 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:24.973326 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:24.973344 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:24.973468 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:24.973515 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 03:56:24.973581 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 03:56:24.976832 25246 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45cd1e66bf834090ba8646a0e79072d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cd1e66bf834090ba8646a0e79072d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43493 } }
I20260808 03:56:24.977186 25246 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45cd1e66bf834090ba8646a0e79072d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:24.977260 25246 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45cd1e66bf834090ba8646a0e79072d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45cd1e66bf834090ba8646a0e79072d9, State: Initialized, Role: FOLLOWER
I20260808 03:56:24.977799 25246 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45cd1e66bf834090ba8646a0e79072d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cd1e66bf834090ba8646a0e79072d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43493 } }
I20260808 03:56:24.978011 25246 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45cd1e66bf834090ba8646a0e79072d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:56:24.978116 25246 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45cd1e66bf834090ba8646a0e79072d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:56:24.978245 25246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45cd1e66bf834090ba8646a0e79072d9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:24.980510 25246 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45cd1e66bf834090ba8646a0e79072d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cd1e66bf834090ba8646a0e79072d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43493 } }
I20260808 03:56:24.980932 25246 leader_election.cc:302] T 00000000000000000000000000000000 P 45cd1e66bf834090ba8646a0e79072d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45cd1e66bf834090ba8646a0e79072d9; no voters:
I20260808 03:56:24.981467 25246 leader_election.cc:288] T 00000000000000000000000000000000 P 45cd1e66bf834090ba8646a0e79072d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:56:24.981586 25259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45cd1e66bf834090ba8646a0e79072d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:24.982112 25259 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45cd1e66bf834090ba8646a0e79072d9 [term 1 LEADER]: Becoming Leader. State: Replica: 45cd1e66bf834090ba8646a0e79072d9, State: Running, Role: LEADER
I20260808 03:56:24.982683 25259 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45cd1e66bf834090ba8646a0e79072d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cd1e66bf834090ba8646a0e79072d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43493 } }
I20260808 03:56:24.984386 25246 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45cd1e66bf834090ba8646a0e79072d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:24.985581 25268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45cd1e66bf834090ba8646a0e79072d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45cd1e66bf834090ba8646a0e79072d9. Latest consensus state: current_term: 1 leader_uuid: "45cd1e66bf834090ba8646a0e79072d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cd1e66bf834090ba8646a0e79072d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43493 } } }
I20260808 03:56:24.985615 25270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45cd1e66bf834090ba8646a0e79072d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45cd1e66bf834090ba8646a0e79072d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cd1e66bf834090ba8646a0e79072d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43493 } } }
I20260808 03:56:24.986002 25270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45cd1e66bf834090ba8646a0e79072d9 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:24.986001 25268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45cd1e66bf834090ba8646a0e79072d9 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:24.988813 25273 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:56:24.992914 25273 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:56:25.002092 25273 catalog_manager.cc:1347] Generated new cluster ID: 50362ede804c4b849be37fc829db108b
I20260808 03:56:25.002202 25273 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:56:25.046813 25273 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:56:25.047242 25273 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:56:25.055147 25273 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45cd1e66bf834090ba8646a0e79072d9: Generated new TSK 0
I20260808 03:56:25.055960 25273 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:56:25.060395 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 25143
W20260808 03:56:25.061360 25238 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51469
W20260808 03:56:25.062284 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 25143: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:56:25.062462 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 25143
2026-08-08T03:56:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2052 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:56:25 dist-test-slave-jlzn krb5kdc[25300](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:56:25 dist-test-slave-jlzn krb5kdc[25300](info): set up 2 sockets
Aug 08 03:56:25 dist-test-slave-jlzn krb5kdc[25300](info): commencing operation
krb5kdc: starting...
W20260808 03:56:26.261725 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:56:26 dist-test-slave-jlzn krb5kdc[25300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:56:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:56:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 03:56:26.562486 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35387
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:26.900820 25319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:26.901141 25319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:26.901280 25319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:26.908571 25319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:26.908708 25319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:26.908788 25319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:26.908879 25319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:26.908937 25319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:26.918977 25319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:26.920372 25319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:26.921720 25319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:26.933126 25330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:26.933208 25326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:26.933182 25327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:26.933914 25319 server_base.cc:1034] running on GCE node
I20260808 03:56:26.934705 25319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:26.936168 25319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:26.937368 25319 hybrid_clock.cc:648] HybridClock initialized: now 1786161386937320 us; error 36 us; skew 500 ppm
I20260808 03:56:26.937781 25319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:26 dist-test-slave-jlzn krb5kdc[25300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161386, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:26.940897 25319 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 03:56:26.942337 25319 webserver.cc:460] Webserver started at http://127.0.0.1:33569/ using document root <none> and password file <none>
I20260808 03:56:26.942994 25319 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:26.943121 25319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:26.943415 25319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:26.946398 25319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "acd083a19d264858b0eade6d9194b742"
format_stamp: "Formatted at 2026-08-08 03:56:26 on dist-test-slave-jlzn"
server_key: "045765dd5e104c76fd65aad6149c439f"
server_key_iv: "a1be9eaadbc80f0cf36810a16032928c"
server_key_version: "encryptionkey@0"
I20260808 03:56:26.947144 25319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "acd083a19d264858b0eade6d9194b742"
format_stamp: "Formatted at 2026-08-08 03:56:26 on dist-test-slave-jlzn"
server_key: "045765dd5e104c76fd65aad6149c439f"
server_key_iv: "a1be9eaadbc80f0cf36810a16032928c"
server_key_version: "encryptionkey@0"
I20260808 03:56:26.952260 25319 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 03:56:26.955605 25337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:26.956899 25319 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:56:26.957067 25319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "acd083a19d264858b0eade6d9194b742"
format_stamp: "Formatted at 2026-08-08 03:56:26 on dist-test-slave-jlzn"
server_key: "045765dd5e104c76fd65aad6149c439f"
server_key_iv: "a1be9eaadbc80f0cf36810a16032928c"
server_key_version: "encryptionkey@0"
I20260808 03:56:26.957254 25319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:26.983290 25319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:26.988986 25319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:26.989319 25319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:27.008186 25319 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35387
I20260808 03:56:27.008232 25421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35387 every 8 connection(s)
I20260808 03:56:27.009536 25319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:56:27.011405 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 25319
I20260808 03:56:27.011626 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:56:27.012245 27210 external_mini_cluster.cc:1428] Setting key 2e7d4ff7743a665cd74f80fc3eb669b5
I20260808 03:56:27.014043 25422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:27.024994 25422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acd083a19d264858b0eade6d9194b742: Bootstrap starting.
Aug 08 03:56:27 dist-test-slave-jlzn krb5kdc[25300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 03:56:27.028640 25422 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acd083a19d264858b0eade6d9194b742: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:27.029695 25422 log.cc:824] T 00000000000000000000000000000000 P acd083a19d264858b0eade6d9194b742: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:27.032505 25422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acd083a19d264858b0eade6d9194b742: No bootstrap required, opened a new log
I20260808 03:56:27.033836 25428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:27.014085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38951 (local address 127.0.0.1:35387)
0808 03:56:27.014859 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:27.014871 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:27.014911 (+ 40us) server_negotiation.cc:407] Connection header received
0808 03:56:27.016585 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:27.016606 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:27.016993 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:27.017355 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:27.018894 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:27.020245 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:27.021452 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:27.022118 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:27.022301 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:27.026475 (+ 4174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:27.026483 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:27.026499 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:27.026534 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:27.029948 (+ 3414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:27.030643 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:27.030653 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:27.030659 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:27.030833 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:27.031213 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:27.031217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:27.031219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:27.031745 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:27.032062 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:27.032517 (+ 455us) server_negotiation.cc:299] Negotiation successful
0808 03:56:27.032908 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":333,"threads_started":1}
I20260808 03:56:27.037397 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:56:27.040480 25422 raft_consensus.cc:348] T 00000000000000000000000000000000 P acd083a19d264858b0eade6d9194b742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd083a19d264858b0eade6d9194b742" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35387 } }
I20260808 03:56:27.040838 25422 raft_consensus.cc:374] T 00000000000000000000000000000000 P acd083a19d264858b0eade6d9194b742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:27.040930 25422 raft_consensus.cc:729] T 00000000000000000000000000000000 P acd083a19d264858b0eade6d9194b742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acd083a19d264858b0eade6d9194b742, State: Initialized, Role: FOLLOWER
I20260808 03:56:27.041571 25422 consensus_queue.cc:260] T 00000000000000000000000000000000 P acd083a19d264858b0eade6d9194b742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd083a19d264858b0eade6d9194b742" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35387 } }
I20260808 03:56:27.041852 25422 raft_consensus.cc:388] T 00000000000000000000000000000000 P acd083a19d264858b0eade6d9194b742 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:56:27.042001 25422 raft_consensus.cc:482] T 00000000000000000000000000000000 P acd083a19d264858b0eade6d9194b742 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:56:27.042193 25422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acd083a19d264858b0eade6d9194b742 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:27.045135 25422 raft_consensus.cc:504] T 00000000000000000000000000000000 P acd083a19d264858b0eade6d9194b742 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd083a19d264858b0eade6d9194b742" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35387 } }
I20260808 03:56:27.045765 25422 leader_election.cc:302] T 00000000000000000000000000000000 P acd083a19d264858b0eade6d9194b742 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: acd083a19d264858b0eade6d9194b742; no voters:
I20260808 03:56:27.046427 25422 leader_election.cc:288] T 00000000000000000000000000000000 P acd083a19d264858b0eade6d9194b742 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:56:27.046528 25439 raft_consensus.cc:2781] T 00000000000000000000000000000000 P acd083a19d264858b0eade6d9194b742 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:27.048820 25422 sys_catalog.cc:564] T 00000000000000000000000000000000 P acd083a19d264858b0eade6d9194b742 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:27.053397 25439 raft_consensus.cc:686] T 00000000000000000000000000000000 P acd083a19d264858b0eade6d9194b742 [term 1 LEADER]: Becoming Leader. State: Replica: acd083a19d264858b0eade6d9194b742, State: Running, Role: LEADER
I20260808 03:56:27.054071 25439 consensus_queue.cc:237] T 00000000000000000000000000000000 P acd083a19d264858b0eade6d9194b742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd083a19d264858b0eade6d9194b742" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35387 } }
I20260808 03:56:27.055477 25428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:27.041468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36143 (local address 127.0.0.1:35387)
0808 03:56:27.041662 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:27.041669 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:27.042058 (+ 389us) server_negotiation.cc:407] Connection header received
0808 03:56:27.042260 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:27.042263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:27.042337 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:27.042412 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:27.044309 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:27.044998 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:27.046584 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:27.047267 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:27.047383 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:27.049768 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:27.049775 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:27.049780 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:27.049812 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:27.052648 (+ 2836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:27.053322 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:27.053328 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:27.053332 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:27.053482 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:27.053890 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:27.053894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:27.053897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:27.054200 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:27.054439 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:27.054796 (+ 357us) server_negotiation.cc:299] Negotiation successful
0808 03:56:27.054996 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 03:56:27.057257 25440 sys_catalog.cc:455] T 00000000000000000000000000000000 P acd083a19d264858b0eade6d9194b742 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "acd083a19d264858b0eade6d9194b742" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd083a19d264858b0eade6d9194b742" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35387 } } }
I20260808 03:56:27.057680 25440 sys_catalog.cc:458] T 00000000000000000000000000000000 P acd083a19d264858b0eade6d9194b742 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:27.058383 25442 sys_catalog.cc:455] T 00000000000000000000000000000000 P acd083a19d264858b0eade6d9194b742 [sys.catalog]: SysCatalogTable state changed. Reason: New leader acd083a19d264858b0eade6d9194b742. Latest consensus state: current_term: 1 leader_uuid: "acd083a19d264858b0eade6d9194b742" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd083a19d264858b0eade6d9194b742" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35387 } } }
I20260808 03:56:27.058673 25442 sys_catalog.cc:458] T 00000000000000000000000000000000 P acd083a19d264858b0eade6d9194b742 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:27.059969 25448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:56:27.065308 25448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:56:27.074962 25448 catalog_manager.cc:1347] Generated new cluster ID: 8be7a8420ef143499506b0ee47702ece
I20260808 03:56:27.075099 25448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:56:27.093216 25448 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:56:27.094609 25448 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:56:27.102173 25448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P acd083a19d264858b0eade6d9194b742: Generated new TSK 0
I20260808 03:56:27.102957 25448 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:56:27.112564 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 25319
I20260808 03:56:27.113834 25414 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:27.202740 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 25319
2026-08-08T03:56:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2141 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:56:27 dist-test-slave-jlzn krb5kdc[25467](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:56:27 dist-test-slave-jlzn krb5kdc[25467](info): set up 2 sockets
Aug 08 03:56:27 dist-test-slave-jlzn krb5kdc[25467](info): commencing operation
krb5kdc: starting...
W20260808 03:56:28.420884 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:56:28 dist-test-slave-jlzn krb5kdc[25467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:56:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:56:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 03:56:28.721083 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45503
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:29.058447 25486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:29.058751 25486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:29.058868 25486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:29.065990 25486 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:29.066100 25486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:29.066145 25486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:29.066206 25486 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:29.066249 25486 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:29.075980 25486 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:29.077173 25486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:29.078400 25486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:29.089593 25494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:29.089637 25497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:29.089663 25495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:29.090464 25486 server_base.cc:1034] running on GCE node
I20260808 03:56:29.091207 25486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:29.092566 25486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:29.093799 25486 hybrid_clock.cc:648] HybridClock initialized: now 1786161389093749 us; error 37 us; skew 500 ppm
I20260808 03:56:29.094167 25486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:29 dist-test-slave-jlzn krb5kdc[25467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161389, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:29.097347 25486 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 03:56:29.098781 25486 webserver.cc:460] Webserver started at http://127.0.0.1:46371/ using document root <none> and password file <none>
I20260808 03:56:29.099429 25486 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:29.099570 25486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:29.099910 25486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:29.102819 25486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "2d92febfec9d47d59bf8e6a70bf4c099"
format_stamp: "Formatted at 2026-08-08 03:56:29 on dist-test-slave-jlzn"
server_key: "85d72cda4ac07cacf1965f72606cd9bf"
server_key_iv: "161f5acd641138c54ff44e0d5821d066"
server_key_version: "encryptionkey@0"
I20260808 03:56:29.103590 25486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "2d92febfec9d47d59bf8e6a70bf4c099"
format_stamp: "Formatted at 2026-08-08 03:56:29 on dist-test-slave-jlzn"
server_key: "85d72cda4ac07cacf1965f72606cd9bf"
server_key_iv: "161f5acd641138c54ff44e0d5821d066"
server_key_version: "encryptionkey@0"
I20260808 03:56:29.108790 25486 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260808 03:56:29.112277 25504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:29.113593 25486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 03:56:29.113762 25486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "2d92febfec9d47d59bf8e6a70bf4c099"
format_stamp: "Formatted at 2026-08-08 03:56:29 on dist-test-slave-jlzn"
server_key: "85d72cda4ac07cacf1965f72606cd9bf"
server_key_iv: "161f5acd641138c54ff44e0d5821d066"
server_key_version: "encryptionkey@0"
I20260808 03:56:29.113951 25486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:29.140380 25486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:29.144800 25486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:29.145083 25486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:29.162278 25486 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45503
I20260808 03:56:29.162309 25583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45503 every 8 connection(s)
I20260808 03:56:29.163596 25486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:56:29.167325 25584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:29.171000 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 25486
I20260808 03:56:29.171219 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:56:29.171782 27210 external_mini_cluster.cc:1428] Setting key affd06f060ea5686dbbc75584a46f395
I20260808 03:56:29.179771 25584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d92febfec9d47d59bf8e6a70bf4c099: Bootstrap starting.
I20260808 03:56:29.183264 25584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d92febfec9d47d59bf8e6a70bf4c099: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:29.184446 25584 log.cc:824] T 00000000000000000000000000000000 P 2d92febfec9d47d59bf8e6a70bf4c099: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:29.184914 25591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:29.173594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47763 (local address 127.0.0.1:45503)
0808 03:56:29.174361 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:29.174372 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:29.174413 (+ 41us) server_negotiation.cc:407] Connection header received
0808 03:56:29.176045 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:29.176066 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:29.176449 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:29.176754 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:29.177344 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:29.178731 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:29.180502 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:29.181221 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:29.181360 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:29.182604 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:29.182612 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:29.182625 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:29.183173 (+ 548us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:29.183257 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:29.183282 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:29.183398 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:29.183606 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 03:56:29.183879 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":353,"threads_started":1}
I20260808 03:56:29.187321 25584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d92febfec9d47d59bf8e6a70bf4c099: No bootstrap required, opened a new log
I20260808 03:56:29.188505 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:56:29.195964 25584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d92febfec9d47d59bf8e6a70bf4c099 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d92febfec9d47d59bf8e6a70bf4c099" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45503 } }
I20260808 03:56:29.196367 25584 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d92febfec9d47d59bf8e6a70bf4c099 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:29.196491 25584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d92febfec9d47d59bf8e6a70bf4c099 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d92febfec9d47d59bf8e6a70bf4c099, State: Initialized, Role: FOLLOWER
I20260808 03:56:29.197194 25584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d92febfec9d47d59bf8e6a70bf4c099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d92febfec9d47d59bf8e6a70bf4c099" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45503 } }
I20260808 03:56:29.197474 25584 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d92febfec9d47d59bf8e6a70bf4c099 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:56:29.197856 25584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d92febfec9d47d59bf8e6a70bf4c099 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:56:29.198067 25584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d92febfec9d47d59bf8e6a70bf4c099 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:29.198833 25591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:29.192765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59871 (local address 127.0.0.1:45503)
0808 03:56:29.192954 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:29.192960 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:29.193225 (+ 265us) server_negotiation.cc:407] Connection header received
0808 03:56:29.193411 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:29.193417 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:29.193496 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:29.193628 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:29.194082 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:29.195171 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:29.196436 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:29.197073 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:29.197195 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:29.198110 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:29.198115 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:29.198118 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:29.198153 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:29.198200 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:29.198221 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:29.198304 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:29.198444 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 03:56:29.198530 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 03:56:29.201021 25584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d92febfec9d47d59bf8e6a70bf4c099 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d92febfec9d47d59bf8e6a70bf4c099" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45503 } }
I20260808 03:56:29.201536 25584 leader_election.cc:302] T 00000000000000000000000000000000 P 2d92febfec9d47d59bf8e6a70bf4c099 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d92febfec9d47d59bf8e6a70bf4c099; no voters:
I20260808 03:56:29.202235 25584 leader_election.cc:288] T 00000000000000000000000000000000 P 2d92febfec9d47d59bf8e6a70bf4c099 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:56:29.202351 25607 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d92febfec9d47d59bf8e6a70bf4c099 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:29.203083 25607 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d92febfec9d47d59bf8e6a70bf4c099 [term 1 LEADER]: Becoming Leader. State: Replica: 2d92febfec9d47d59bf8e6a70bf4c099, State: Running, Role: LEADER
I20260808 03:56:29.203732 25607 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d92febfec9d47d59bf8e6a70bf4c099 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d92febfec9d47d59bf8e6a70bf4c099" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45503 } }
I20260808 03:56:29.204723 25584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d92febfec9d47d59bf8e6a70bf4c099 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:29.206787 25609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d92febfec9d47d59bf8e6a70bf4c099 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d92febfec9d47d59bf8e6a70bf4c099. Latest consensus state: current_term: 1 leader_uuid: "2d92febfec9d47d59bf8e6a70bf4c099" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d92febfec9d47d59bf8e6a70bf4c099" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45503 } } }
I20260808 03:56:29.206820 25608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d92febfec9d47d59bf8e6a70bf4c099 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d92febfec9d47d59bf8e6a70bf4c099" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d92febfec9d47d59bf8e6a70bf4c099" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45503 } } }
I20260808 03:56:29.207329 25609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d92febfec9d47d59bf8e6a70bf4c099 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:29.207329 25608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d92febfec9d47d59bf8e6a70bf4c099 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:29.213466 25614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:56:29.218513 25614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:56:29.226806 25614 catalog_manager.cc:1347] Generated new cluster ID: 7d7cd56464814b21a00dbe93dbc2e477
I20260808 03:56:29.226918 25614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:56:29.246089 25614 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:56:29.247993 25614 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:56:29.256875 25614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d92febfec9d47d59bf8e6a70bf4c099: Generated new TSK 0
I20260808 03:56:29.257906 25614 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:56:29.270010 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 25486
W20260808 03:56:29.271031 25572 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47763
W20260808 03:56:29.271749 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 25486: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:56:29.271924 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 25486
2026-08-08T03:56:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2073 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:56:29 dist-test-slave-jlzn krb5kdc[25633](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:56:29 dist-test-slave-jlzn krb5kdc[25633](info): set up 2 sockets
Aug 08 03:56:29 dist-test-slave-jlzn krb5kdc[25633](info): commencing operation
krb5kdc: starting...
W20260808 03:56:30.507048 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.002s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:56:30 dist-test-slave-jlzn krb5kdc[25633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:56:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:56:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 03:56:30.802613 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45377
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:31.137941 25651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:31.138235 25651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:31.138357 25651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:31.145445 25651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:31.145550 25651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:31.145622 25651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:31.145699 25651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:31.145759 25651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:31.155521 25651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:31.156793 25651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:31.158094 25651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:31.169796 25662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:31.169865 25663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:31.169844 25665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:31.170974 25651 server_base.cc:1034] running on GCE node
I20260808 03:56:31.171761 25651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:31.173158 25651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:31.174363 25651 hybrid_clock.cc:648] HybridClock initialized: now 1786161391174316 us; error 39 us; skew 500 ppm
I20260808 03:56:31.174715 25651 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:31 dist-test-slave-jlzn krb5kdc[25633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161391, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:31.177901 25651 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 03:56:31.179458 25651 webserver.cc:460] Webserver started at http://127.0.0.1:39557/ using document root <none> and password file <none>
I20260808 03:56:31.180164 25651 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:31.180295 25651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:31.180606 25651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:31.183548 25651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "1ab71ae312114371b4184f37d36120bb"
format_stamp: "Formatted at 2026-08-08 03:56:31 on dist-test-slave-jlzn"
server_key: "95bdeebcf0d3f22ae32aa10ea76876c0"
server_key_iv: "df1a561b111c081b502273fdaaecdfc4"
server_key_version: "encryptionkey@0"
I20260808 03:56:31.184347 25651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "1ab71ae312114371b4184f37d36120bb"
format_stamp: "Formatted at 2026-08-08 03:56:31 on dist-test-slave-jlzn"
server_key: "95bdeebcf0d3f22ae32aa10ea76876c0"
server_key_iv: "df1a561b111c081b502273fdaaecdfc4"
server_key_version: "encryptionkey@0"
I20260808 03:56:31.189404 25651 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 03:56:31.192894 25676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:31.194187 25651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:56:31.194350 25651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "1ab71ae312114371b4184f37d36120bb"
format_stamp: "Formatted at 2026-08-08 03:56:31 on dist-test-slave-jlzn"
server_key: "95bdeebcf0d3f22ae32aa10ea76876c0"
server_key_iv: "df1a561b111c081b502273fdaaecdfc4"
server_key_version: "encryptionkey@0"
I20260808 03:56:31.194536 25651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:31.200393 25651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:31.204264 25651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:31.204530 25651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:31.221839 25651 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45377
I20260808 03:56:31.221874 25743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45377 every 8 connection(s)
I20260808 03:56:31.223196 25651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:56:31.227109 25744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:31.231673 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 25651
I20260808 03:56:31.231894 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:56:31.232481 27210 external_mini_cluster.cc:1428] Setting key bf97c496daf9d800c9008b248d425cea
I20260808 03:56:31.238178 25744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ab71ae312114371b4184f37d36120bb: Bootstrap starting.
I20260808 03:56:31.240258 25748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:31.234287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54617 (local address 127.0.0.1:45377)
0808 03:56:31.235081 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:31.235093 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:31.235137 (+ 44us) server_negotiation.cc:407] Connection header received
0808 03:56:31.236963 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:31.236985 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:31.237297 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:31.237636 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:31.238682 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:31.238688 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:31.238701 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:31.238715 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:31.238773 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:31.238792 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:31.238989 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:31.239126 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 03:56:31.239310 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":334,"threads_started":1}
I20260808 03:56:31.241278 25744 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ab71ae312114371b4184f37d36120bb: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:31.242276 25744 log.cc:824] T 00000000000000000000000000000000 P 1ab71ae312114371b4184f37d36120bb: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:31.243885 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:56:31.245473 25744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ab71ae312114371b4184f37d36120bb: No bootstrap required, opened a new log
I20260808 03:56:31.251021 25748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:31.248802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51463 (local address 127.0.0.1:45377)
0808 03:56:31.248965 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:31.248975 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:31.249276 (+ 301us) server_negotiation.cc:407] Connection header received
0808 03:56:31.249439 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:31.249443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:31.249489 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:31.249566 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:31.250424 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:31.250429 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:31.250433 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:31.250451 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:31.250487 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:31.250501 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:31.250614 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:31.250699 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 03:56:31.250758 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 03:56:31.253921 25744 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ab71ae312114371b4184f37d36120bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab71ae312114371b4184f37d36120bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45377 } }
I20260808 03:56:31.254261 25744 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ab71ae312114371b4184f37d36120bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:31.254354 25744 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ab71ae312114371b4184f37d36120bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ab71ae312114371b4184f37d36120bb, State: Initialized, Role: FOLLOWER
I20260808 03:56:31.254899 25744 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ab71ae312114371b4184f37d36120bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab71ae312114371b4184f37d36120bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45377 } }
I20260808 03:56:31.255131 25744 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ab71ae312114371b4184f37d36120bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:56:31.255259 25744 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ab71ae312114371b4184f37d36120bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:56:31.255409 25744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ab71ae312114371b4184f37d36120bb [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:31.257527 25744 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ab71ae312114371b4184f37d36120bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab71ae312114371b4184f37d36120bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45377 } }
I20260808 03:56:31.258042 25744 leader_election.cc:302] T 00000000000000000000000000000000 P 1ab71ae312114371b4184f37d36120bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ab71ae312114371b4184f37d36120bb; no voters:
I20260808 03:56:31.258647 25744 leader_election.cc:288] T 00000000000000000000000000000000 P 1ab71ae312114371b4184f37d36120bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:56:31.258738 25757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ab71ae312114371b4184f37d36120bb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:31.259464 25757 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ab71ae312114371b4184f37d36120bb [term 1 LEADER]: Becoming Leader. State: Replica: 1ab71ae312114371b4184f37d36120bb, State: Running, Role: LEADER
I20260808 03:56:31.260077 25757 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ab71ae312114371b4184f37d36120bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab71ae312114371b4184f37d36120bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45377 } }
I20260808 03:56:31.260923 25744 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ab71ae312114371b4184f37d36120bb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:31.263406 25758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ab71ae312114371b4184f37d36120bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ab71ae312114371b4184f37d36120bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab71ae312114371b4184f37d36120bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45377 } } }
I20260808 03:56:31.263830 25758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ab71ae312114371b4184f37d36120bb [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:31.263672 25760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ab71ae312114371b4184f37d36120bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ab71ae312114371b4184f37d36120bb. Latest consensus state: current_term: 1 leader_uuid: "1ab71ae312114371b4184f37d36120bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab71ae312114371b4184f37d36120bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45377 } } }
I20260808 03:56:31.263994 25760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ab71ae312114371b4184f37d36120bb [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:31.266172 25766 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:56:31.270198 25766 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:56:31.277941 25766 catalog_manager.cc:1347] Generated new cluster ID: fda51fc9af33433ea40869202c8cf557
I20260808 03:56:31.278054 25766 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:56:31.285231 25766 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:56:31.285590 25766 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:56:31.297787 25766 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ab71ae312114371b4184f37d36120bb: Generated new TSK 0
I20260808 03:56:31.298530 25766 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:56:31.308135 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 25651
W20260808 03:56:31.309278 25738 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54617
W20260808 03:56:31.310015 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 25651: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:56:31.310186 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 25651
2026-08-08T03:56:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2033 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:56:31 dist-test-slave-jlzn krb5kdc[25789](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:56:31 dist-test-slave-jlzn krb5kdc[25789](info): set up 2 sockets
Aug 08 03:56:31 dist-test-slave-jlzn krb5kdc[25789](info): commencing operation
krb5kdc: starting...
W20260808 03:56:32.506441 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:56:32 dist-test-slave-jlzn krb5kdc[25789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:56:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:56:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 03:56:32.807956 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36409
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:33.146003 25805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:33.146306 25805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:33.146459 25805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:33.153529 25805 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:33.153653 25805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:33.153734 25805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:33.153812 25805 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:33.153872 25805 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:33.163285 25805 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:33.164541 25805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:33.165800 25805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:33.177091 25813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:33.177052 25812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:33.177121 25818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:33.178028 25805 server_base.cc:1034] running on GCE node
I20260808 03:56:33.178759 25805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:33.180127 25805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:33.181327 25805 hybrid_clock.cc:648] HybridClock initialized: now 1786161393181271 us; error 51 us; skew 500 ppm
I20260808 03:56:33.181653 25805 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:33 dist-test-slave-jlzn krb5kdc[25789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161393, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:33.184719 25805 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 03:56:33.186239 25805 webserver.cc:460] Webserver started at http://127.0.0.1:39163/ using document root <none> and password file <none>
I20260808 03:56:33.186882 25805 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:33.187001 25805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:33.187299 25805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:33.190290 25805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "9e57b981e5fc4d16a7e6d11be4a3643f"
format_stamp: "Formatted at 2026-08-08 03:56:33 on dist-test-slave-jlzn"
server_key: "ce4888e5b11a185c684e13cd5b473426"
server_key_iv: "2a20f50e3037e6e558dfea888b7f0faa"
server_key_version: "encryptionkey@0"
I20260808 03:56:33.191053 25805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "9e57b981e5fc4d16a7e6d11be4a3643f"
format_stamp: "Formatted at 2026-08-08 03:56:33 on dist-test-slave-jlzn"
server_key: "ce4888e5b11a185c684e13cd5b473426"
server_key_iv: "2a20f50e3037e6e558dfea888b7f0faa"
server_key_version: "encryptionkey@0"
I20260808 03:56:33.196058 25805 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 03:56:33.199319 25827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:33.200580 25805 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:56:33.200744 25805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "9e57b981e5fc4d16a7e6d11be4a3643f"
format_stamp: "Formatted at 2026-08-08 03:56:33 on dist-test-slave-jlzn"
server_key: "ce4888e5b11a185c684e13cd5b473426"
server_key_iv: "2a20f50e3037e6e558dfea888b7f0faa"
server_key_version: "encryptionkey@0"
I20260808 03:56:33.200915 25805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:33.230154 25805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:33.234982 25805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:33.235253 25805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:33.253281 25805 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36409
I20260808 03:56:33.253321 25897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36409 every 8 connection(s)
I20260808 03:56:33.254626 25805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:56:33.258343 25899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:33.258651 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 25805
I20260808 03:56:33.258811 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:56:33.259256 27210 external_mini_cluster.cc:1428] Setting key e462a2cf9b303276426439e7716d1e0c
I20260808 03:56:33.270308 25899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e57b981e5fc4d16a7e6d11be4a3643f: Bootstrap starting.
Aug 08 03:56:33 dist-test-slave-jlzn krb5kdc[25789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 03:56:33.273500 25899 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e57b981e5fc4d16a7e6d11be4a3643f: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:33.274498 25899 log.cc:824] T 00000000000000000000000000000000 P 9e57b981e5fc4d16a7e6d11be4a3643f: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:33.277796 25899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e57b981e5fc4d16a7e6d11be4a3643f: No bootstrap required, opened a new log
I20260808 03:56:33.282454 25902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:33.261174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:50721 (local address 127.0.0.1:36409)
0808 03:56:33.261878 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:33.261889 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:33.261927 (+ 38us) server_negotiation.cc:407] Connection header received
0808 03:56:33.264235 (+ 2308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:33.264257 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:33.264644 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:33.264995 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:33.266406 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:33.267704 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:33.268986 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:33.269643 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:33.269801 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:33.274219 (+ 4418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:33.274227 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:33.274244 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:33.274280 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:33.277852 (+ 3572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:33.278641 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:33.278649 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:33.278658 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:33.278870 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:33.279298 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:33.279303 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:33.279310 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:33.280078 (+ 768us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:33.280404 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:33.280784 (+ 380us) server_negotiation.cc:299] Negotiation successful
0808 03:56:33.281128 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":298,"threads_started":1}
I20260808 03:56:33.286369 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:56:33.286947 25899 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e57b981e5fc4d16a7e6d11be4a3643f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e57b981e5fc4d16a7e6d11be4a3643f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36409 } }
I20260808 03:56:33.287369 25899 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e57b981e5fc4d16a7e6d11be4a3643f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:33.287463 25899 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e57b981e5fc4d16a7e6d11be4a3643f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e57b981e5fc4d16a7e6d11be4a3643f, State: Initialized, Role: FOLLOWER
I20260808 03:56:33.288096 25899 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e57b981e5fc4d16a7e6d11be4a3643f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e57b981e5fc4d16a7e6d11be4a3643f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36409 } }
I20260808 03:56:33.288367 25899 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e57b981e5fc4d16a7e6d11be4a3643f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:56:33.288501 25899 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e57b981e5fc4d16a7e6d11be4a3643f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:56:33.288663 25899 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e57b981e5fc4d16a7e6d11be4a3643f [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:33.291642 25899 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e57b981e5fc4d16a7e6d11be4a3643f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e57b981e5fc4d16a7e6d11be4a3643f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36409 } }
I20260808 03:56:33.292236 25899 leader_election.cc:302] T 00000000000000000000000000000000 P 9e57b981e5fc4d16a7e6d11be4a3643f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e57b981e5fc4d16a7e6d11be4a3643f; no voters:
I20260808 03:56:33.292881 25899 leader_election.cc:288] T 00000000000000000000000000000000 P 9e57b981e5fc4d16a7e6d11be4a3643f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:56:33.292974 25912 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e57b981e5fc4d16a7e6d11be4a3643f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:33.295524 25899 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e57b981e5fc4d16a7e6d11be4a3643f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:33.299604 25912 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e57b981e5fc4d16a7e6d11be4a3643f [term 1 LEADER]: Becoming Leader. State: Replica: 9e57b981e5fc4d16a7e6d11be4a3643f, State: Running, Role: LEADER
I20260808 03:56:33.300267 25912 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e57b981e5fc4d16a7e6d11be4a3643f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e57b981e5fc4d16a7e6d11be4a3643f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36409 } }
I20260808 03:56:33.304108 25913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e57b981e5fc4d16a7e6d11be4a3643f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e57b981e5fc4d16a7e6d11be4a3643f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e57b981e5fc4d16a7e6d11be4a3643f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36409 } } }
I20260808 03:56:33.304483 25913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e57b981e5fc4d16a7e6d11be4a3643f [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:33.307339 25902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:33.291507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:38613 (local address 127.0.0.1:36409)
0808 03:56:33.291703 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:33.291709 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:33.292103 (+ 394us) server_negotiation.cc:407] Connection header received
0808 03:56:33.292269 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:33.292275 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:33.292352 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:33.292457 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:33.294358 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:33.295212 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:33.296525 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:33.297096 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:33.297221 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:33.299605 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:33.299610 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:33.299614 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:33.299649 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:33.302947 (+ 3298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:33.303964 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:33.303970 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:33.303975 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:33.304159 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:33.304887 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:33.304893 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:33.304897 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:33.305351 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:33.305629 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:33.306773 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0808 03:56:33.306962 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 03:56:33.311707 25914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e57b981e5fc4d16a7e6d11be4a3643f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e57b981e5fc4d16a7e6d11be4a3643f. Latest consensus state: current_term: 1 leader_uuid: "9e57b981e5fc4d16a7e6d11be4a3643f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e57b981e5fc4d16a7e6d11be4a3643f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36409 } } }
I20260808 03:56:33.312227 25914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e57b981e5fc4d16a7e6d11be4a3643f [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:33.312502 25923 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:56:33.317693 25923 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:56:33.326141 25923 catalog_manager.cc:1347] Generated new cluster ID: 30832b18abcc45c090d298a437549f99
I20260808 03:56:33.326238 25923 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:56:33.344935 25923 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:56:33.346181 25923 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:56:33.358994 25923 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e57b981e5fc4d16a7e6d11be4a3643f: Generated new TSK 0
I20260808 03:56:33.359841 25923 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:56:33.369019 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 25805
I20260808 03:56:33.370628 25891 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:33.461656 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 25805
W20260808 03:56:33.482627 25785 connection.cc:535] client connection to 127.0.0.1:36409 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36409): Connection reset by peer (error 104)
2026-08-08T03:56:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2148 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:56:33 dist-test-slave-jlzn krb5kdc[25939](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:56:33 dist-test-slave-jlzn krb5kdc[25939](info): set up 2 sockets
Aug 08 03:56:33 dist-test-slave-jlzn krb5kdc[25939](info): commencing operation
krb5kdc: starting...
W20260808 03:56:34.666121 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:56:34 dist-test-slave-jlzn krb5kdc[25939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:56:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:56:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 03:56:34.977454 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37259
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:35.309630 25956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:35.309907 25956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:35.309998 25956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:35.316962 25956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:35.317085 25956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:35.317131 25956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:35.317193 25956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:35.317237 25956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:35.326917 25956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:35.328173 25956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:35.329493 25956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:35.341310 25964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:35.341378 25961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:35.341357 25962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:35.341861 25956 server_base.cc:1034] running on GCE node
I20260808 03:56:35.342705 25956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:35.344029 25956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:35.345235 25956 hybrid_clock.cc:648] HybridClock initialized: now 1786161395345189 us; error 36 us; skew 500 ppm
I20260808 03:56:35.345561 25956 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:35 dist-test-slave-jlzn krb5kdc[25939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161395, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:35.348590 25956 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 03:56:35.349928 25956 webserver.cc:460] Webserver started at http://127.0.0.1:40945/ using document root <none> and password file <none>
I20260808 03:56:35.350541 25956 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:35.350641 25956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:35.350890 25956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:35.353857 25956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "ecc7042c4e9345779ffea891a0f6dea8"
format_stamp: "Formatted at 2026-08-08 03:56:35 on dist-test-slave-jlzn"
server_key: "38eb476fc8200e7f4751f3a86ed1c95e"
server_key_iv: "c63705cc83d337a36fafbd0246cd388b"
server_key_version: "encryptionkey@0"
I20260808 03:56:35.354615 25956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "ecc7042c4e9345779ffea891a0f6dea8"
format_stamp: "Formatted at 2026-08-08 03:56:35 on dist-test-slave-jlzn"
server_key: "38eb476fc8200e7f4751f3a86ed1c95e"
server_key_iv: "c63705cc83d337a36fafbd0246cd388b"
server_key_version: "encryptionkey@0"
I20260808 03:56:35.359560 25956 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260808 03:56:35.362996 25973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:35.364312 25956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:56:35.364464 25956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "ecc7042c4e9345779ffea891a0f6dea8"
format_stamp: "Formatted at 2026-08-08 03:56:35 on dist-test-slave-jlzn"
server_key: "38eb476fc8200e7f4751f3a86ed1c95e"
server_key_iv: "c63705cc83d337a36fafbd0246cd388b"
server_key_version: "encryptionkey@0"
I20260808 03:56:35.364635 25956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:35.399518 25956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:35.403764 25956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:35.404063 25956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:35.421149 25956 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37259
I20260808 03:56:35.421187 26047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37259 every 8 connection(s)
I20260808 03:56:35.422442 25956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:56:35.426095 26048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:35.426585 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 25956
I20260808 03:56:35.426776 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:56:35.427367 27210 external_mini_cluster.cc:1428] Setting key 12c16d45e20a24556d7bd98244fbe374
I20260808 03:56:35.439390 26048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecc7042c4e9345779ffea891a0f6dea8: Bootstrap starting.
I20260808 03:56:35.440927 26052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:35.429122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:42335 (local address 127.0.0.1:37259)
0808 03:56:35.429840 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:35.429851 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:35.429894 (+ 43us) server_negotiation.cc:407] Connection header received
0808 03:56:35.432098 (+ 2204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:35.432120 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:35.432498 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:35.432826 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:35.433385 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:35.434604 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:35.436474 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:35.437235 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:35.437369 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:35.438678 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:35.438686 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:35.438700 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:35.439217 (+ 517us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:35.439303 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:35.439328 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:35.439449 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:35.439726 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 03:56:35.439944 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":313,"threads_started":1}
I20260808 03:56:35.442601 26048 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecc7042c4e9345779ffea891a0f6dea8: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:35.443713 26048 log.cc:824] T 00000000000000000000000000000000 P ecc7042c4e9345779ffea891a0f6dea8: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:35.444517 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:56:35.446401 26048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecc7042c4e9345779ffea891a0f6dea8: No bootstrap required, opened a new log
I20260808 03:56:35.454135 26048 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecc7042c4e9345779ffea891a0f6dea8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc7042c4e9345779ffea891a0f6dea8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37259 } }
I20260808 03:56:35.454507 26048 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecc7042c4e9345779ffea891a0f6dea8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:35.454665 26048 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecc7042c4e9345779ffea891a0f6dea8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecc7042c4e9345779ffea891a0f6dea8, State: Initialized, Role: FOLLOWER
I20260808 03:56:35.454876 26052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:35.448755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:54739 (local address 127.0.0.1:37259)
0808 03:56:35.448911 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:35.448916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:35.449417 (+ 501us) server_negotiation.cc:407] Connection header received
0808 03:56:35.449625 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:35.449629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:35.449691 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:35.449784 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:35.450332 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:35.451120 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:35.452446 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:35.452945 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:35.453061 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:35.454111 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:35.454115 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:35.454119 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:35.454140 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:35.454191 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:35.454206 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:35.454299 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:35.454483 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 03:56:35.454536 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 03:56:35.455293 26048 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecc7042c4e9345779ffea891a0f6dea8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc7042c4e9345779ffea891a0f6dea8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37259 } }
I20260808 03:56:35.455596 26048 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecc7042c4e9345779ffea891a0f6dea8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:56:35.455749 26048 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecc7042c4e9345779ffea891a0f6dea8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:56:35.455960 26048 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecc7042c4e9345779ffea891a0f6dea8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:35.458495 26048 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecc7042c4e9345779ffea891a0f6dea8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc7042c4e9345779ffea891a0f6dea8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37259 } }
I20260808 03:56:35.458988 26048 leader_election.cc:302] T 00000000000000000000000000000000 P ecc7042c4e9345779ffea891a0f6dea8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecc7042c4e9345779ffea891a0f6dea8; no voters:
I20260808 03:56:35.459523 26048 leader_election.cc:288] T 00000000000000000000000000000000 P ecc7042c4e9345779ffea891a0f6dea8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:56:35.459621 26059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecc7042c4e9345779ffea891a0f6dea8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:35.460320 26059 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecc7042c4e9345779ffea891a0f6dea8 [term 1 LEADER]: Becoming Leader. State: Replica: ecc7042c4e9345779ffea891a0f6dea8, State: Running, Role: LEADER
I20260808 03:56:35.460883 26059 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecc7042c4e9345779ffea891a0f6dea8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc7042c4e9345779ffea891a0f6dea8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37259 } }
I20260808 03:56:35.461509 26048 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecc7042c4e9345779ffea891a0f6dea8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:35.463407 26060 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecc7042c4e9345779ffea891a0f6dea8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecc7042c4e9345779ffea891a0f6dea8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc7042c4e9345779ffea891a0f6dea8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37259 } } }
I20260808 03:56:35.463788 26060 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecc7042c4e9345779ffea891a0f6dea8 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:35.465725 26065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:56:35.465436 26062 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecc7042c4e9345779ffea891a0f6dea8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecc7042c4e9345779ffea891a0f6dea8. Latest consensus state: current_term: 1 leader_uuid: "ecc7042c4e9345779ffea891a0f6dea8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc7042c4e9345779ffea891a0f6dea8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37259 } } }
I20260808 03:56:35.465831 26062 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecc7042c4e9345779ffea891a0f6dea8 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:35.470691 26065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:56:35.479946 26065 catalog_manager.cc:1347] Generated new cluster ID: 665f199a477849099cbeef5006ec792d
I20260808 03:56:35.480057 26065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:56:35.509615 26065 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:56:35.510955 26065 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:56:35.517607 26065 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecc7042c4e9345779ffea891a0f6dea8: Generated new TSK 0
I20260808 03:56:35.518396 26065 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:56:35.529237 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 25956
W20260808 03:56:35.530277 26040 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.68:42335
W20260808 03:56:35.531009 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 25956: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:56:35.531132 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 25956
W20260808 03:56:35.556962 25937 connection.cc:535] client connection to 127.0.0.1:37259 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37259): Connection reset by peer (error 104)
2026-08-08T03:56:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2074 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:56:35 dist-test-slave-jlzn krb5kdc[26088](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:56:35 dist-test-slave-jlzn krb5kdc[26088](info): set up 2 sockets
Aug 08 03:56:35 dist-test-slave-jlzn krb5kdc[26088](info): commencing operation
krb5kdc: starting...
W20260808 03:56:36.748296 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.001s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:56:36 dist-test-slave-jlzn krb5kdc[26088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:56:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:56:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 03:56:37.068332 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33553
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:37.405122 26102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:37.405404 26102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:37.405504 26102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:37.412722 26102 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:37.412838 26102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:37.412892 26102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:37.412952 26102 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:37.412997 26102 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:37.422847 26102 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:37.424181 26102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:37.425493 26102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:37.437394 26108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:37.437315 26112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:37.437351 26109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:37.438167 26102 server_base.cc:1034] running on GCE node
I20260808 03:56:37.438939 26102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:37.440416 26102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:37.441636 26102 hybrid_clock.cc:648] HybridClock initialized: now 1786161397441578 us; error 50 us; skew 500 ppm
I20260808 03:56:37.441977 26102 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:37 dist-test-slave-jlzn krb5kdc[26088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161397, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:37.445052 26102 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 03:56:37.446440 26102 webserver.cc:460] Webserver started at http://127.0.0.1:43735/ using document root <none> and password file <none>
I20260808 03:56:37.447201 26102 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:37.447305 26102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:37.447553 26102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:37.450544 26102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "cdf4efc76f8a423fa90800b01d76ce24"
format_stamp: "Formatted at 2026-08-08 03:56:37 on dist-test-slave-jlzn"
server_key: "68b310edf2881a0edc23a7789bbd3c1d"
server_key_iv: "9a97a3e4b962ed8f1c37e4680275f442"
server_key_version: "encryptionkey@0"
I20260808 03:56:37.451287 26102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "cdf4efc76f8a423fa90800b01d76ce24"
format_stamp: "Formatted at 2026-08-08 03:56:37 on dist-test-slave-jlzn"
server_key: "68b310edf2881a0edc23a7789bbd3c1d"
server_key_iv: "9a97a3e4b962ed8f1c37e4680275f442"
server_key_version: "encryptionkey@0"
I20260808 03:56:37.456358 26102 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260808 03:56:37.459636 26123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:37.460896 26102 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:56:37.461040 26102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "cdf4efc76f8a423fa90800b01d76ce24"
format_stamp: "Formatted at 2026-08-08 03:56:37 on dist-test-slave-jlzn"
server_key: "68b310edf2881a0edc23a7789bbd3c1d"
server_key_iv: "9a97a3e4b962ed8f1c37e4680275f442"
server_key_version: "encryptionkey@0"
I20260808 03:56:37.461225 26102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:37.466677 26102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:37.470486 26102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:37.470726 26102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:37.488044 26102 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33553
I20260808 03:56:37.488085 26191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33553 every 8 connection(s)
I20260808 03:56:37.489374 26102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:56:37.493067 26194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:37.497908 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 26102
I20260808 03:56:37.498065 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:56:37.498494 27210 external_mini_cluster.cc:1428] Setting key 42993ac7d8a23024f6098d52b1971637
I20260808 03:56:37.504937 26194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdf4efc76f8a423fa90800b01d76ce24: Bootstrap starting.
I20260808 03:56:37.507364 26197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:37.500330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:60985 (local address 127.0.0.1:33553)
0808 03:56:37.501174 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:37.501189 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:37.501241 (+ 52us) server_negotiation.cc:407] Connection header received
0808 03:56:37.503626 (+ 2385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:37.503654 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:37.504096 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:37.504511 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:37.505463 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:37.505472 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:37.505488 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:37.505513 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:37.505578 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:37.505605 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:37.505815 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:37.505936 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 03:56:37.506128 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":330,"threads_started":1}
I20260808 03:56:37.508637 26194 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdf4efc76f8a423fa90800b01d76ce24: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:37.509684 26194 log.cc:824] T 00000000000000000000000000000000 P cdf4efc76f8a423fa90800b01d76ce24: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:37.510913 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:56:37.512771 26194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdf4efc76f8a423fa90800b01d76ce24: No bootstrap required, opened a new log
I20260808 03:56:37.518352 26197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:37.516073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:56759 (local address 127.0.0.1:33553)
0808 03:56:37.516250 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:37.516256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:37.516606 (+ 350us) server_negotiation.cc:407] Connection header received
0808 03:56:37.516742 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:37.516748 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:37.516808 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:37.516919 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:37.517756 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:37.517762 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:37.517766 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:37.517785 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:37.517828 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:37.517849 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:37.517966 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:37.518007 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 03:56:37.518060 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 03:56:37.521651 26194 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdf4efc76f8a423fa90800b01d76ce24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdf4efc76f8a423fa90800b01d76ce24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33553 } }
I20260808 03:56:37.521975 26194 raft_consensus.cc:374] T 00000000000000000000000000000000 P cdf4efc76f8a423fa90800b01d76ce24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:37.522048 26194 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdf4efc76f8a423fa90800b01d76ce24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdf4efc76f8a423fa90800b01d76ce24, State: Initialized, Role: FOLLOWER
I20260808 03:56:37.522509 26194 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdf4efc76f8a423fa90800b01d76ce24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdf4efc76f8a423fa90800b01d76ce24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33553 } }
I20260808 03:56:37.522732 26194 raft_consensus.cc:388] T 00000000000000000000000000000000 P cdf4efc76f8a423fa90800b01d76ce24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:56:37.522827 26194 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdf4efc76f8a423fa90800b01d76ce24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:56:37.522954 26194 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdf4efc76f8a423fa90800b01d76ce24 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:37.525208 26194 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdf4efc76f8a423fa90800b01d76ce24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdf4efc76f8a423fa90800b01d76ce24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33553 } }
I20260808 03:56:37.525658 26194 leader_election.cc:302] T 00000000000000000000000000000000 P cdf4efc76f8a423fa90800b01d76ce24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cdf4efc76f8a423fa90800b01d76ce24; no voters:
I20260808 03:56:37.526134 26194 leader_election.cc:288] T 00000000000000000000000000000000 P cdf4efc76f8a423fa90800b01d76ce24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:56:37.526304 26207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cdf4efc76f8a423fa90800b01d76ce24 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:37.526902 26207 raft_consensus.cc:686] T 00000000000000000000000000000000 P cdf4efc76f8a423fa90800b01d76ce24 [term 1 LEADER]: Becoming Leader. State: Replica: cdf4efc76f8a423fa90800b01d76ce24, State: Running, Role: LEADER
I20260808 03:56:37.527504 26207 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdf4efc76f8a423fa90800b01d76ce24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdf4efc76f8a423fa90800b01d76ce24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33553 } }
I20260808 03:56:37.528267 26194 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdf4efc76f8a423fa90800b01d76ce24 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:37.530118 26208 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdf4efc76f8a423fa90800b01d76ce24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cdf4efc76f8a423fa90800b01d76ce24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdf4efc76f8a423fa90800b01d76ce24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33553 } } }
I20260808 03:56:37.530567 26208 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdf4efc76f8a423fa90800b01d76ce24 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:37.534580 26209 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdf4efc76f8a423fa90800b01d76ce24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdf4efc76f8a423fa90800b01d76ce24. Latest consensus state: current_term: 1 leader_uuid: "cdf4efc76f8a423fa90800b01d76ce24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdf4efc76f8a423fa90800b01d76ce24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33553 } } }
I20260808 03:56:37.534943 26209 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdf4efc76f8a423fa90800b01d76ce24 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:37.535902 26217 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:56:37.540827 26217 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:56:37.549397 26217 catalog_manager.cc:1347] Generated new cluster ID: 1c3acc72c4df4d4aa601c0319f1626f4
I20260808 03:56:37.549503 26217 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:56:37.561849 26217 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:56:37.562352 26217 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:56:37.569840 26217 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cdf4efc76f8a423fa90800b01d76ce24: Generated new TSK 0
I20260808 03:56:37.570823 26217 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:56:37.585238 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 26102
W20260808 03:56:37.586810 26186 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.68:60985
W20260808 03:56:37.587538 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 26102: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:56:37.587702 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 26102
2026-08-08T03:56:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2057 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:56:37 dist-test-slave-jlzn krb5kdc[26230](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:56:37 dist-test-slave-jlzn krb5kdc[26230](info): set up 2 sockets
Aug 08 03:56:37 dist-test-slave-jlzn krb5kdc[26230](info): commencing operation
krb5kdc: starting...
W20260808 03:56:38.811209 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:56:39 dist-test-slave-jlzn krb5kdc[26230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:56:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:56:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 03:56:39.111268 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43739
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:39.445004 26252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:39.445317 26252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:39.445458 26252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:39.452596 26252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:39.452715 26252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:39.452793 26252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:39.452877 26252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:39.452936 26252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:39.463021 26252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43739
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:39.464347 26252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:39.465570 26252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:39.476794 26259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:39.476891 26257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:39.476941 26264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:39.477880 26252 server_base.cc:1034] running on GCE node
I20260808 03:56:39.478623 26252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:39.479977 26252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:39.481187 26252 hybrid_clock.cc:648] HybridClock initialized: now 1786161399481140 us; error 46 us; skew 500 ppm
I20260808 03:56:39.481503 26252 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:39 dist-test-slave-jlzn krb5kdc[26230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161399, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:39.484639 26252 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 03:56:39.486002 26252 webserver.cc:460] Webserver started at http://127.0.0.1:34123/ using document root <none> and password file <none>
I20260808 03:56:39.486654 26252 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:39.486774 26252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:39.487063 26252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:39.490024 26252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "8e56d99c55ea46a794d909cd047beaf1"
format_stamp: "Formatted at 2026-08-08 03:56:39 on dist-test-slave-jlzn"
server_key: "0c499c155da670e6f72f8d4b3daf6078"
server_key_iv: "ce741dfbd64f96f08400e95b9f8f17b6"
server_key_version: "encryptionkey@0"
I20260808 03:56:39.490792 26252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "8e56d99c55ea46a794d909cd047beaf1"
format_stamp: "Formatted at 2026-08-08 03:56:39 on dist-test-slave-jlzn"
server_key: "0c499c155da670e6f72f8d4b3daf6078"
server_key_iv: "ce741dfbd64f96f08400e95b9f8f17b6"
server_key_version: "encryptionkey@0"
I20260808 03:56:39.495705 26252 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 03:56:39.499055 26272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:39.500339 26252 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 03:56:39.500501 26252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "8e56d99c55ea46a794d909cd047beaf1"
format_stamp: "Formatted at 2026-08-08 03:56:39 on dist-test-slave-jlzn"
server_key: "0c499c155da670e6f72f8d4b3daf6078"
server_key_iv: "ce741dfbd64f96f08400e95b9f8f17b6"
server_key_version: "encryptionkey@0"
I20260808 03:56:39.500696 26252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:39.516254 26252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:39.520133 26252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:39.520387 26252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:39.536741 26252 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43739
I20260808 03:56:39.536774 26344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43739 every 8 connection(s)
I20260808 03:56:39.538166 26252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:56:39.539204 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 26252
I20260808 03:56:39.539383 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:56:39.539845 27210 external_mini_cluster.cc:1428] Setting key 2663b63f778c5accdd05a76117854a52
I20260808 03:56:39.542536 26346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 03:56:39 dist-test-slave-jlzn krb5kdc[26230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 03:56:39.554862 26346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e56d99c55ea46a794d909cd047beaf1: Bootstrap starting.
I20260808 03:56:39.558730 26346 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e56d99c55ea46a794d909cd047beaf1: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:39.560014 26346 log.cc:824] T 00000000000000000000000000000000 P 8e56d99c55ea46a794d909cd047beaf1: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:39.563269 26346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e56d99c55ea46a794d909cd047beaf1: No bootstrap required, opened a new log
I20260808 03:56:39.564222 26351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:39.541666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:47145 (local address 127.0.0.1:43739)
0808 03:56:39.542426 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:39.542441 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:39.542493 (+ 52us) server_negotiation.cc:407] Connection header received
0808 03:56:39.544770 (+ 2277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:39.544798 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:39.545273 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:39.545728 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:39.547059 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:39.549110 (+ 2051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:39.550387 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:39.551394 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:39.551588 (+ 194us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:39.555581 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:39.555593 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:39.555610 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:39.555659 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:39.559491 (+ 3832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:39.560358 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:39.560365 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:39.560373 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:39.560591 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:39.561144 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:39.561147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:39.561149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:39.561777 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:39.562076 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:39.562521 (+ 445us) server_negotiation.cc:299] Negotiation successful
0808 03:56:39.562843 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":314,"threads_started":1}
I20260808 03:56:39.568131 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:56:39.571911 26346 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e56d99c55ea46a794d909cd047beaf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e56d99c55ea46a794d909cd047beaf1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43739 } }
I20260808 03:56:39.572295 26346 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e56d99c55ea46a794d909cd047beaf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:39.572402 26346 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e56d99c55ea46a794d909cd047beaf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e56d99c55ea46a794d909cd047beaf1, State: Initialized, Role: FOLLOWER
I20260808 03:56:39.573063 26346 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e56d99c55ea46a794d909cd047beaf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e56d99c55ea46a794d909cd047beaf1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43739 } }
I20260808 03:56:39.573335 26346 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e56d99c55ea46a794d909cd047beaf1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:56:39.573493 26346 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e56d99c55ea46a794d909cd047beaf1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:56:39.573684 26346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e56d99c55ea46a794d909cd047beaf1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:39.576520 26346 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e56d99c55ea46a794d909cd047beaf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e56d99c55ea46a794d909cd047beaf1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43739 } }
I20260808 03:56:39.577304 26346 leader_election.cc:302] T 00000000000000000000000000000000 P 8e56d99c55ea46a794d909cd047beaf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e56d99c55ea46a794d909cd047beaf1; no voters:
I20260808 03:56:39.577999 26346 leader_election.cc:288] T 00000000000000000000000000000000 P 8e56d99c55ea46a794d909cd047beaf1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:56:39.578119 26360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e56d99c55ea46a794d909cd047beaf1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:39.579216 26360 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e56d99c55ea46a794d909cd047beaf1 [term 1 LEADER]: Becoming Leader. State: Replica: 8e56d99c55ea46a794d909cd047beaf1, State: Running, Role: LEADER
I20260808 03:56:39.580189 26346 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e56d99c55ea46a794d909cd047beaf1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:39.580168 26360 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e56d99c55ea46a794d909cd047beaf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e56d99c55ea46a794d909cd047beaf1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43739 } }
I20260808 03:56:39.585777 26361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e56d99c55ea46a794d909cd047beaf1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e56d99c55ea46a794d909cd047beaf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e56d99c55ea46a794d909cd047beaf1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43739 } } }
I20260808 03:56:39.586179 26361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e56d99c55ea46a794d909cd047beaf1 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:39.586719 26362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e56d99c55ea46a794d909cd047beaf1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e56d99c55ea46a794d909cd047beaf1. Latest consensus state: current_term: 1 leader_uuid: "8e56d99c55ea46a794d909cd047beaf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e56d99c55ea46a794d909cd047beaf1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43739 } } }
I20260808 03:56:39.587021 26362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e56d99c55ea46a794d909cd047beaf1 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:39.590646 26351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:39.572233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:34637 (local address 127.0.0.1:43739)
0808 03:56:39.572441 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:39.572446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:39.572938 (+ 492us) server_negotiation.cc:407] Connection header received
0808 03:56:39.573129 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:39.573133 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:39.573191 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:39.573268 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:39.575095 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:39.575929 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:39.577716 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:39.578380 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:39.578529 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:39.581890 (+ 3361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:39.581897 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:39.581903 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:39.581945 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:39.586009 (+ 4064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:39.587162 (+ 1153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:39.587168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:39.587173 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:39.587419 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:39.588097 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:39.588102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:39.588106 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:39.588493 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:39.589884 (+ 1391us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:39.590068 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 03:56:39.590259 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 03:56:39.592295 26369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:56:39.596802 26369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:56:39.605041 26369 catalog_manager.cc:1347] Generated new cluster ID: 5e415a4b2bf847ea9e87deae03fae450
I20260808 03:56:39.605150 26369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:56:39.611745 26369 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:56:39.613449 26369 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:56:39.621863 26369 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e56d99c55ea46a794d909cd047beaf1: Generated new TSK 0
I20260808 03:56:39.622650 26369 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:56:39.627722 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 26252
I20260808 03:56:39.629184 26338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:39.714846 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 26252
W20260808 03:56:39.737493 26228 connection.cc:535] client connection to 127.0.0.1:43739 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43739): Connection reset by peer (error 104)
2026-08-08T03:56:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2123 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:56:39 dist-test-slave-jlzn krb5kdc[26392](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:56:39 dist-test-slave-jlzn krb5kdc[26392](info): set up 2 sockets
Aug 08 03:56:39 dist-test-slave-jlzn krb5kdc[26392](info): commencing operation
krb5kdc: starting...
W20260808 03:56:40.931596 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:56:41 dist-test-slave-jlzn krb5kdc[26392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:56:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:56:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 03:56:41.230671 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45551
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:41.565465 26412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:41.565759 26412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:41.565881 26412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:41.573148 26412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:41.573262 26412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:41.573347 26412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:41.573443 26412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:41.573522 26412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:41.583207 26412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45551
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:41.584540 26412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:41.585834 26412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:41.597385 26421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:41.597476 26422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:41.597708 26424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:41.598776 26412 server_base.cc:1034] running on GCE node
I20260808 03:56:41.599608 26412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:41.601064 26412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:41.602276 26412 hybrid_clock.cc:648] HybridClock initialized: now 1786161401602232 us; error 34 us; skew 500 ppm
I20260808 03:56:41.602677 26412 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:41 dist-test-slave-jlzn krb5kdc[26392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161401, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:41.605830 26412 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 03:56:41.607286 26412 webserver.cc:460] Webserver started at http://127.0.0.1:41185/ using document root <none> and password file <none>
I20260808 03:56:41.608017 26412 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:41.608142 26412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:41.608430 26412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:41.611460 26412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "c38c18dbfa01494a9841cb38738d15bd"
format_stamp: "Formatted at 2026-08-08 03:56:41 on dist-test-slave-jlzn"
server_key: "f539a44735c77762c1d35b633a0ec8d2"
server_key_iv: "4d9d0bf9f912ae5cafd250d9981b3fc6"
server_key_version: "encryptionkey@0"
I20260808 03:56:41.612275 26412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "c38c18dbfa01494a9841cb38738d15bd"
format_stamp: "Formatted at 2026-08-08 03:56:41 on dist-test-slave-jlzn"
server_key: "f539a44735c77762c1d35b633a0ec8d2"
server_key_iv: "4d9d0bf9f912ae5cafd250d9981b3fc6"
server_key_version: "encryptionkey@0"
I20260808 03:56:41.617435 26412 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 03:56:41.620929 26431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:41.622290 26412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 03:56:41.622547 26412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "c38c18dbfa01494a9841cb38738d15bd"
format_stamp: "Formatted at 2026-08-08 03:56:41 on dist-test-slave-jlzn"
server_key: "f539a44735c77762c1d35b633a0ec8d2"
server_key_iv: "4d9d0bf9f912ae5cafd250d9981b3fc6"
server_key_version: "encryptionkey@0"
I20260808 03:56:41.622748 26412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:41.650826 26412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:41.654901 26412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:41.655185 26412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:41.672780 26412 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45551
I20260808 03:56:41.672811 26495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45551 every 8 connection(s)
I20260808 03:56:41.674141 26412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:56:41.677932 26496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:41.679240 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 26412
I20260808 03:56:41.679401 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:56:41.679929 27210 external_mini_cluster.cc:1428] Setting key df138e6d1fed5d48ebf971491024e2f8
I20260808 03:56:41.692317 26496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c38c18dbfa01494a9841cb38738d15bd: Bootstrap starting.
I20260808 03:56:41.694278 26499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:41.681668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:37267 (local address 127.0.0.1:45551)
0808 03:56:41.682463 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:41.682478 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:41.682543 (+ 65us) server_negotiation.cc:407] Connection header received
0808 03:56:41.684337 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:41.684365 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:41.684888 (+ 523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:41.685334 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:41.685921 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:41.687774 (+ 1853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:41.689099 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:41.690158 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:41.690327 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:41.691542 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:41.691553 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:41.691571 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:41.692279 (+ 708us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:41.692389 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:41.692423 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:41.692591 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:41.692843 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 03:56:41.693013 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":318,"threads_started":1}
I20260808 03:56:41.695904 26496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c38c18dbfa01494a9841cb38738d15bd: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:41.697070 26496 log.cc:824] T 00000000000000000000000000000000 P c38c18dbfa01494a9841cb38738d15bd: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:41.698067 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:56:41.700325 26496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c38c18dbfa01494a9841cb38738d15bd: No bootstrap required, opened a new log
I20260808 03:56:41.709748 26499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:41.703248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:57767 (local address 127.0.0.1:45551)
0808 03:56:41.703430 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:41.703436 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:41.703759 (+ 323us) server_negotiation.cc:407] Connection header received
0808 03:56:41.703927 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:41.703933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:41.704014 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:41.704124 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:41.704557 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:41.705688 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:41.707028 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:41.707721 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:41.707867 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:41.708987 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:41.708994 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:41.708999 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:41.709019 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:41.709079 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:41.709099 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:41.709205 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:41.709386 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 03:56:41.709440 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 03:56:41.709851 26496 raft_consensus.cc:348] T 00000000000000000000000000000000 P c38c18dbfa01494a9841cb38738d15bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c38c18dbfa01494a9841cb38738d15bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45551 } }
I20260808 03:56:41.710260 26496 raft_consensus.cc:374] T 00000000000000000000000000000000 P c38c18dbfa01494a9841cb38738d15bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:41.710376 26496 raft_consensus.cc:729] T 00000000000000000000000000000000 P c38c18dbfa01494a9841cb38738d15bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c38c18dbfa01494a9841cb38738d15bd, State: Initialized, Role: FOLLOWER
I20260808 03:56:41.710891 26496 consensus_queue.cc:260] T 00000000000000000000000000000000 P c38c18dbfa01494a9841cb38738d15bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c38c18dbfa01494a9841cb38738d15bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45551 } }
I20260808 03:56:41.711135 26496 raft_consensus.cc:388] T 00000000000000000000000000000000 P c38c18dbfa01494a9841cb38738d15bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:56:41.711270 26496 raft_consensus.cc:482] T 00000000000000000000000000000000 P c38c18dbfa01494a9841cb38738d15bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:56:41.711421 26496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c38c18dbfa01494a9841cb38738d15bd [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:41.713616 26496 raft_consensus.cc:504] T 00000000000000000000000000000000 P c38c18dbfa01494a9841cb38738d15bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c38c18dbfa01494a9841cb38738d15bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45551 } }
I20260808 03:56:41.714087 26496 leader_election.cc:302] T 00000000000000000000000000000000 P c38c18dbfa01494a9841cb38738d15bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c38c18dbfa01494a9841cb38738d15bd; no voters:
I20260808 03:56:41.714643 26496 leader_election.cc:288] T 00000000000000000000000000000000 P c38c18dbfa01494a9841cb38738d15bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:56:41.714792 26508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c38c18dbfa01494a9841cb38738d15bd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:41.715446 26508 raft_consensus.cc:686] T 00000000000000000000000000000000 P c38c18dbfa01494a9841cb38738d15bd [term 1 LEADER]: Becoming Leader. State: Replica: c38c18dbfa01494a9841cb38738d15bd, State: Running, Role: LEADER
I20260808 03:56:41.715997 26508 consensus_queue.cc:237] T 00000000000000000000000000000000 P c38c18dbfa01494a9841cb38738d15bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c38c18dbfa01494a9841cb38738d15bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45551 } }
I20260808 03:56:41.716572 26496 sys_catalog.cc:564] T 00000000000000000000000000000000 P c38c18dbfa01494a9841cb38738d15bd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:41.718629 26509 sys_catalog.cc:455] T 00000000000000000000000000000000 P c38c18dbfa01494a9841cb38738d15bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c38c18dbfa01494a9841cb38738d15bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c38c18dbfa01494a9841cb38738d15bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45551 } } }
I20260808 03:56:41.718978 26509 sys_catalog.cc:458] T 00000000000000000000000000000000 P c38c18dbfa01494a9841cb38738d15bd [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:41.718544 26510 sys_catalog.cc:455] T 00000000000000000000000000000000 P c38c18dbfa01494a9841cb38738d15bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader c38c18dbfa01494a9841cb38738d15bd. Latest consensus state: current_term: 1 leader_uuid: "c38c18dbfa01494a9841cb38738d15bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c38c18dbfa01494a9841cb38738d15bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45551 } } }
I20260808 03:56:41.719309 26510 sys_catalog.cc:458] T 00000000000000000000000000000000 P c38c18dbfa01494a9841cb38738d15bd [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:41.720603 26518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:56:41.725677 26518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:56:41.734674 26518 catalog_manager.cc:1347] Generated new cluster ID: 59b68883cbd947c68fed05dee83b2c54
I20260808 03:56:41.734772 26518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:56:41.753197 26518 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:56:41.755153 26518 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:56:41.766858 26518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c38c18dbfa01494a9841cb38738d15bd: Generated new TSK 0
I20260808 03:56:41.767710 26518 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:56:41.778512 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 26412
W20260808 03:56:41.779690 26491 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.68:37267
W20260808 03:56:41.780478 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 26412: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:56:41.780642 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 26412
W20260808 03:56:41.804860 26389 connection.cc:535] client connection to 127.0.0.1:45551 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45551): Connection reset by peer (error 104)
2026-08-08T03:56:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2066 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:56:41 dist-test-slave-jlzn krb5kdc[26538](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:56:41 dist-test-slave-jlzn krb5kdc[26538](info): set up 2 sockets
Aug 08 03:56:41 dist-test-slave-jlzn krb5kdc[26538](info): commencing operation
krb5kdc: starting...
W20260808 03:56:43.009835 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.003s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:56:43 dist-test-slave-jlzn krb5kdc[26538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:56:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:56:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 03:56:43.321483 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38305
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:43.659442 26556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:43.659752 26556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:43.659909 26556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:43.667043 26556 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:43.667162 26556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:43.667234 26556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:43.667315 26556 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:43.667375 26556 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:43.677245 26556 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38305
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:43.678531 26556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:43.679893 26556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:43.691336 26561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:43.691479 26565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:43.691411 26562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:43.692065 26556 server_base.cc:1034] running on GCE node
I20260808 03:56:43.692822 26556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:43.694167 26556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:43.695374 26556 hybrid_clock.cc:648] HybridClock initialized: now 1786161403695327 us; error 35 us; skew 500 ppm
I20260808 03:56:43.695711 26556 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:43 dist-test-slave-jlzn krb5kdc[26538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161403, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:43.698833 26556 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 03:56:43.700294 26556 webserver.cc:460] Webserver started at http://127.0.0.1:37107/ using document root <none> and password file <none>
I20260808 03:56:43.700939 26556 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:43.701059 26556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:43.701364 26556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:43.704456 26556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "575020c614fa403e8b7fb4fdb3222fae"
format_stamp: "Formatted at 2026-08-08 03:56:43 on dist-test-slave-jlzn"
server_key: "cb3956849926c2ab9e81e4d7b8bf9de1"
server_key_iv: "9d52018e91f384be0478219b5825852e"
server_key_version: "encryptionkey@0"
I20260808 03:56:43.705236 26556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "575020c614fa403e8b7fb4fdb3222fae"
format_stamp: "Formatted at 2026-08-08 03:56:43 on dist-test-slave-jlzn"
server_key: "cb3956849926c2ab9e81e4d7b8bf9de1"
server_key_iv: "9d52018e91f384be0478219b5825852e"
server_key_version: "encryptionkey@0"
I20260808 03:56:43.710366 26556 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:56:43.714097 26575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:43.715346 26556 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:56:43.715512 26556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "575020c614fa403e8b7fb4fdb3222fae"
format_stamp: "Formatted at 2026-08-08 03:56:43 on dist-test-slave-jlzn"
server_key: "cb3956849926c2ab9e81e4d7b8bf9de1"
server_key_iv: "9d52018e91f384be0478219b5825852e"
server_key_version: "encryptionkey@0"
I20260808 03:56:43.715701 26556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:43.721406 26556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:43.725508 26556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:43.725783 26556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:43.744071 26556 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38305
I20260808 03:56:43.744140 26644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38305 every 8 connection(s)
I20260808 03:56:43.745546 26556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:56:43.749366 26645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:43.752759 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 26556
I20260808 03:56:43.752944 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:56:43.753504 27210 external_mini_cluster.cc:1428] Setting key e1137caeb30ce881b4abcefd9295b7cb
I20260808 03:56:43.761266 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:43.755494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:51315 (local address 127.0.0.1:38305)
0808 03:56:43.756262 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:43.756274 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:43.756316 (+ 42us) server_negotiation.cc:407] Connection header received
0808 03:56:43.757958 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:43.757981 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:43.758314 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:43.758670 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:43.759697 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:43.759704 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:43.759716 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:43.759730 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:43.759796 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:43.759849 (+ 53us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:43.760052 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:43.760186 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 03:56:43.760367 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":309,"threads_started":1}
I20260808 03:56:43.761579 26645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 575020c614fa403e8b7fb4fdb3222fae: Bootstrap starting.
I20260808 03:56:43.765096 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:56:43.765266 26645 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 575020c614fa403e8b7fb4fdb3222fae: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:43.766463 26645 log.cc:824] T 00000000000000000000000000000000 P 575020c614fa403e8b7fb4fdb3222fae: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:43.769734 26645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 575020c614fa403e8b7fb4fdb3222fae: No bootstrap required, opened a new log
I20260808 03:56:43.772563 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:43.770038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:58821 (local address 127.0.0.1:38305)
0808 03:56:43.770232 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:43.770239 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:43.770606 (+ 367us) server_negotiation.cc:407] Connection header received
0808 03:56:43.770829 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:43.770834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:43.770889 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:43.770978 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:43.771917 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:43.771921 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:43.771924 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:43.771940 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:43.771982 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:43.772018 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:43.772118 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:43.772230 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 03:56:43.772279 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 03:56:43.778331 26645 raft_consensus.cc:348] T 00000000000000000000000000000000 P 575020c614fa403e8b7fb4fdb3222fae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575020c614fa403e8b7fb4fdb3222fae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38305 } }
I20260808 03:56:43.778668 26645 raft_consensus.cc:374] T 00000000000000000000000000000000 P 575020c614fa403e8b7fb4fdb3222fae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:43.778755 26645 raft_consensus.cc:729] T 00000000000000000000000000000000 P 575020c614fa403e8b7fb4fdb3222fae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575020c614fa403e8b7fb4fdb3222fae, State: Initialized, Role: FOLLOWER
I20260808 03:56:43.779281 26645 consensus_queue.cc:260] T 00000000000000000000000000000000 P 575020c614fa403e8b7fb4fdb3222fae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575020c614fa403e8b7fb4fdb3222fae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38305 } }
I20260808 03:56:43.779496 26645 raft_consensus.cc:388] T 00000000000000000000000000000000 P 575020c614fa403e8b7fb4fdb3222fae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:56:43.779623 26645 raft_consensus.cc:482] T 00000000000000000000000000000000 P 575020c614fa403e8b7fb4fdb3222fae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:56:43.779769 26645 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 575020c614fa403e8b7fb4fdb3222fae [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:43.781924 26645 raft_consensus.cc:504] T 00000000000000000000000000000000 P 575020c614fa403e8b7fb4fdb3222fae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575020c614fa403e8b7fb4fdb3222fae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38305 } }
I20260808 03:56:43.782421 26645 leader_election.cc:302] T 00000000000000000000000000000000 P 575020c614fa403e8b7fb4fdb3222fae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 575020c614fa403e8b7fb4fdb3222fae; no voters:
I20260808 03:56:43.782963 26645 leader_election.cc:288] T 00000000000000000000000000000000 P 575020c614fa403e8b7fb4fdb3222fae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:56:43.783098 26657 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 575020c614fa403e8b7fb4fdb3222fae [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:43.783829 26657 raft_consensus.cc:686] T 00000000000000000000000000000000 P 575020c614fa403e8b7fb4fdb3222fae [term 1 LEADER]: Becoming Leader. State: Replica: 575020c614fa403e8b7fb4fdb3222fae, State: Running, Role: LEADER
I20260808 03:56:43.784387 26657 consensus_queue.cc:237] T 00000000000000000000000000000000 P 575020c614fa403e8b7fb4fdb3222fae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575020c614fa403e8b7fb4fdb3222fae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38305 } }
I20260808 03:56:43.785056 26645 sys_catalog.cc:564] T 00000000000000000000000000000000 P 575020c614fa403e8b7fb4fdb3222fae [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:43.786965 26658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 575020c614fa403e8b7fb4fdb3222fae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "575020c614fa403e8b7fb4fdb3222fae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575020c614fa403e8b7fb4fdb3222fae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38305 } } }
I20260808 03:56:43.787358 26658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 575020c614fa403e8b7fb4fdb3222fae [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:43.787892 26659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 575020c614fa403e8b7fb4fdb3222fae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 575020c614fa403e8b7fb4fdb3222fae. Latest consensus state: current_term: 1 leader_uuid: "575020c614fa403e8b7fb4fdb3222fae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575020c614fa403e8b7fb4fdb3222fae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38305 } } }
I20260808 03:56:43.788271 26659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 575020c614fa403e8b7fb4fdb3222fae [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:43.789520 26667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:56:43.794844 26667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:56:43.804989 26667 catalog_manager.cc:1347] Generated new cluster ID: a85799fed7db48e0b71796f1684f86eb
I20260808 03:56:43.805116 26667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:56:43.846271 26667 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:56:43.846787 26667 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:56:43.860989 26667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 575020c614fa403e8b7fb4fdb3222fae: Generated new TSK 0
I20260808 03:56:43.861837 26667 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:56:43.872851 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 26556
W20260808 03:56:43.874274 26640 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.68:51315
W20260808 03:56:43.874990 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 26556: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:56:43.875123 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 26556
2026-08-08T03:56:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2099 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:56:43 dist-test-slave-jlzn krb5kdc[26686](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:56:43 dist-test-slave-jlzn krb5kdc[26686](info): set up 2 sockets
Aug 08 03:56:43 dist-test-slave-jlzn krb5kdc[26686](info): commencing operation
krb5kdc: starting...
W20260808 03:56:45.095877 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.003s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:56:45 dist-test-slave-jlzn krb5kdc[26686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:56:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:56:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:56:45.397136 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:35601
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:41041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:35601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:45.735703 26706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:45.736048 26706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:45.736166 26706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:45.743340 26706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:45.743459 26706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:45.743506 26706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:45.743566 26706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:45.743610 26706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:45.754654 26706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:41041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:35601
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:35601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:45.756043 26706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:45.757388 26706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:45.768937 26713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:45.768975 26714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:45.769008 26717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:45.769698 26706 server_base.cc:1034] running on GCE node
I20260808 03:56:45.770514 26706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:45.771968 26706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:45.773187 26706 hybrid_clock.cc:648] HybridClock initialized: now 1786161405773148 us; error 29 us; skew 500 ppm
I20260808 03:56:45.773531 26706 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:45 dist-test-slave-jlzn krb5kdc[26686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161405, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:45.781329 26706 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:56:45.782914 26706 webserver.cc:460] Webserver started at http://127.26.146.190:38429/ using document root <none> and password file <none>
I20260808 03:56:45.783607 26706 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:45.783733 26706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:45.784067 26706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:45.787421 26706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "bb1c3379e90b4f6c821fe63fb3518cfa"
format_stamp: "Formatted at 2026-08-08 03:56:45 on dist-test-slave-jlzn"
server_key: "582506dad174c68162db71d1a2aa2a25"
server_key_iv: "4a7f56b9d872588ec016d73abebcc64b"
server_key_version: "encryptionkey@0"
I20260808 03:56:45.788262 26706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "bb1c3379e90b4f6c821fe63fb3518cfa"
format_stamp: "Formatted at 2026-08-08 03:56:45 on dist-test-slave-jlzn"
server_key: "582506dad174c68162db71d1a2aa2a25"
server_key_iv: "4a7f56b9d872588ec016d73abebcc64b"
server_key_version: "encryptionkey@0"
I20260808 03:56:45.793624 26706 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260808 03:56:45.797238 26729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:45.798623 26706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:56:45.798794 26706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "bb1c3379e90b4f6c821fe63fb3518cfa"
format_stamp: "Formatted at 2026-08-08 03:56:45 on dist-test-slave-jlzn"
server_key: "582506dad174c68162db71d1a2aa2a25"
server_key_iv: "4a7f56b9d872588ec016d73abebcc64b"
server_key_version: "encryptionkey@0"
I20260808 03:56:45.798980 26706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:45.842247 26706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:45.846524 26706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:45.846829 26706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:45.864909 26706 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:35601
I20260808 03:56:45.864929 26801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:35601 every 8 connection(s)
I20260808 03:56:45.866276 26706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:56:45.867776 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 26706
I20260808 03:56:45.867998 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:56:45.868579 27210 external_mini_cluster.cc:1428] Setting key 720f2cf0fb5eecab48f15bfb8880000f
I20260808 03:56:45.870963 26805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 03:56:45 dist-test-slave-jlzn krb5kdc[26686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:56:45.884115 26805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb1c3379e90b4f6c821fe63fb3518cfa: Bootstrap starting.
I20260808 03:56:45.887506 26805 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb1c3379e90b4f6c821fe63fb3518cfa: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:45.888648 26805 log.cc:824] T 00000000000000000000000000000000 P bb1c3379e90b4f6c821fe63fb3518cfa: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:45.892004 26805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb1c3379e90b4f6c821fe63fb3518cfa: No bootstrap required, opened a new log
I20260808 03:56:45.892860 26808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:45.870371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43601 (local address 127.26.146.190:35601)
0808 03:56:45.871112 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:45.871124 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:45.871165 (+ 41us) server_negotiation.cc:407] Connection header received
0808 03:56:45.872819 (+ 1654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:45.872842 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:45.873208 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:45.873544 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:45.875422 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:45.876990 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:45.878764 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:45.879752 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:45.879951 (+ 199us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:45.885351 (+ 5400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:45.885359 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:45.885373 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:45.885412 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:45.888455 (+ 3043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:45.889447 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:45.889456 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:45.889465 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:45.889751 (+ 286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:45.890168 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:45.890175 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:45.890177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:45.890701 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:45.890926 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:45.891439 (+ 513us) server_negotiation.cc:299] Negotiation successful
0808 03:56:45.891775 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":305,"threads_started":1}
I20260808 03:56:45.896646 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:56:45.899834 26805 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb1c3379e90b4f6c821fe63fb3518cfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1c3379e90b4f6c821fe63fb3518cfa" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 35601 } }
I20260808 03:56:45.900228 26805 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb1c3379e90b4f6c821fe63fb3518cfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:45.900318 26805 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb1c3379e90b4f6c821fe63fb3518cfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb1c3379e90b4f6c821fe63fb3518cfa, State: Initialized, Role: FOLLOWER
I20260808 03:56:45.900939 26805 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb1c3379e90b4f6c821fe63fb3518cfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1c3379e90b4f6c821fe63fb3518cfa" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 35601 } }
I20260808 03:56:45.901271 26805 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb1c3379e90b4f6c821fe63fb3518cfa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:56:45.901415 26805 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb1c3379e90b4f6c821fe63fb3518cfa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:56:45.901582 26805 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb1c3379e90b4f6c821fe63fb3518cfa [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:45.904055 26805 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb1c3379e90b4f6c821fe63fb3518cfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1c3379e90b4f6c821fe63fb3518cfa" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 35601 } }
I20260808 03:56:45.904543 26805 leader_election.cc:302] T 00000000000000000000000000000000 P bb1c3379e90b4f6c821fe63fb3518cfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb1c3379e90b4f6c821fe63fb3518cfa; no voters:
I20260808 03:56:45.905155 26805 leader_election.cc:288] T 00000000000000000000000000000000 P bb1c3379e90b4f6c821fe63fb3518cfa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:56:45.905265 26817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb1c3379e90b4f6c821fe63fb3518cfa [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:45.906293 26817 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb1c3379e90b4f6c821fe63fb3518cfa [term 1 LEADER]: Becoming Leader. State: Replica: bb1c3379e90b4f6c821fe63fb3518cfa, State: Running, Role: LEADER
I20260808 03:56:45.906901 26817 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb1c3379e90b4f6c821fe63fb3518cfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1c3379e90b4f6c821fe63fb3518cfa" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 35601 } }
I20260808 03:56:45.907405 26805 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb1c3379e90b4f6c821fe63fb3518cfa [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:45.910003 26818 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb1c3379e90b4f6c821fe63fb3518cfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb1c3379e90b4f6c821fe63fb3518cfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1c3379e90b4f6c821fe63fb3518cfa" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 35601 } } }
I20260808 03:56:45.910404 26818 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb1c3379e90b4f6c821fe63fb3518cfa [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:45.911207 26819 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb1c3379e90b4f6c821fe63fb3518cfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb1c3379e90b4f6c821fe63fb3518cfa. Latest consensus state: current_term: 1 leader_uuid: "bb1c3379e90b4f6c821fe63fb3518cfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1c3379e90b4f6c821fe63fb3518cfa" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 35601 } } }
I20260808 03:56:45.911518 26819 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb1c3379e90b4f6c821fe63fb3518cfa [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:45.918330 26808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:45.901042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60775 (local address 127.26.146.190:35601)
0808 03:56:45.901230 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:45.901237 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:45.901630 (+ 393us) server_negotiation.cc:407] Connection header received
0808 03:56:45.901806 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:45.901811 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:45.901878 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:45.901957 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:45.903542 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:45.904381 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:45.906064 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:45.906750 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:45.906889 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:45.910001 (+ 3112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:45.910007 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:45.910011 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:45.910052 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:45.913737 (+ 3685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:45.914812 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:45.914819 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:45.914823 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:45.915037 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:45.915618 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:45.915624 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:45.915628 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:45.916063 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:45.916380 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:45.917734 (+ 1354us) server_negotiation.cc:299] Negotiation successful
0808 03:56:45.917937 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 03:56:45.920717 26823 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:56:45.926936 26823 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:56:45.937237 26823 catalog_manager.cc:1347] Generated new cluster ID: bd8b7f57fdb34a5f809a93ec54d614f8
I20260808 03:56:45.937362 26823 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:56:45.956327 26823 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:56:45.957626 26823 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:56:45.965713 26823 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb1c3379e90b4f6c821fe63fb3518cfa: Generated new TSK 0
I20260808 03:56:45.966574 26823 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:56:45.978440 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 26706
I20260808 03:56:45.979698 26796 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:46.070188 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 26706
W20260808 03:56:46.093449 26684 connection.cc:535] client connection to 127.26.146.190:35601 recv error: Network error: failed to read from TLS socket (remote: 127.26.146.190:35601): Connection reset by peer (error 104)
2026-08-08T03:56:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2189 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:56:46 dist-test-slave-jlzn krb5kdc[26845](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:56:46 dist-test-slave-jlzn krb5kdc[26845](info): set up 2 sockets
Aug 08 03:56:46 dist-test-slave-jlzn krb5kdc[26845](info): commencing operation
krb5kdc: starting...
W20260808 03:56:47.282776 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.001s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:56:47 dist-test-slave-jlzn krb5kdc[26845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:56:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:56:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:56:47.593102 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:43647
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:35425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:43647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:47.937873 26863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:47.938180 26863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:47.938292 26863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:47.945390 26863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:47.945499 26863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:47.945551 26863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:47.945611 26863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:47.945655 26863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:47.955581 26863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:35425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:43647
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:43647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:47.956874 26863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:47.958158 26863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:47.969537 26868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:47.969448 26869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:47.969496 26871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:47.970358 26863 server_base.cc:1034] running on GCE node
I20260808 03:56:47.971086 26863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:47.972537 26863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:47.973745 26863 hybrid_clock.cc:648] HybridClock initialized: now 1786161407973690 us; error 41 us; skew 500 ppm
I20260808 03:56:47.974079 26863 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:47 dist-test-slave-jlzn krb5kdc[26845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161407, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:47.977358 26863 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:56:47.978842 26863 webserver.cc:460] Webserver started at http://127.26.146.190:32921/ using document root <none> and password file <none>
I20260808 03:56:47.979499 26863 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:47.979636 26863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:47.979967 26863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:47.983052 26863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "dbb45df842f346ebb62f8b342f293499"
format_stamp: "Formatted at 2026-08-08 03:56:47 on dist-test-slave-jlzn"
server_key: "b3ac0b7168604e0ef76659471546e5ec"
server_key_iv: "678b3752ce6020c67e09c2731745a9f6"
server_key_version: "encryptionkey@0"
I20260808 03:56:47.983896 26863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "dbb45df842f346ebb62f8b342f293499"
format_stamp: "Formatted at 2026-08-08 03:56:47 on dist-test-slave-jlzn"
server_key: "b3ac0b7168604e0ef76659471546e5ec"
server_key_iv: "678b3752ce6020c67e09c2731745a9f6"
server_key_version: "encryptionkey@0"
I20260808 03:56:47.989130 26863 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 03:56:47.992650 26881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:47.993985 26863 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:56:47.994153 26863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "dbb45df842f346ebb62f8b342f293499"
format_stamp: "Formatted at 2026-08-08 03:56:47 on dist-test-slave-jlzn"
server_key: "b3ac0b7168604e0ef76659471546e5ec"
server_key_iv: "678b3752ce6020c67e09c2731745a9f6"
server_key_version: "encryptionkey@0"
I20260808 03:56:47.994331 26863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:48.015305 26863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:48.019770 26863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:48.020081 26863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:48.038023 26863 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:43647
I20260808 03:56:48.038055 26946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:43647 every 8 connection(s)
I20260808 03:56:48.039357 26863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:56:48.043370 26947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:48.044482 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 26863
I20260808 03:56:48.044698 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:56:48.045128 27210 external_mini_cluster.cc:1428] Setting key 9986215b424a6424dd4c736d3f6ccfc6
I20260808 03:56:48.057860 26947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbb45df842f346ebb62f8b342f293499: Bootstrap starting.
I20260808 03:56:48.059834 26952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:48.046984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34709 (local address 127.26.146.190:43647)
0808 03:56:48.047824 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:48.047840 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:48.047898 (+ 58us) server_negotiation.cc:407] Connection header received
0808 03:56:48.049814 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:48.049845 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:48.050370 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:48.050808 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:48.051331 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:48.053227 (+ 1896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:48.054628 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:48.055710 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:48.055914 (+ 204us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:48.057068 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:48.057080 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:48.057099 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:48.057794 (+ 695us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:48.057900 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:48.057933 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:48.058088 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:48.058356 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 03:56:48.058525 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":305,"threads_started":1}
I20260808 03:56:48.061805 26947 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbb45df842f346ebb62f8b342f293499: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:48.063004 26947 log.cc:824] T 00000000000000000000000000000000 P dbb45df842f346ebb62f8b342f293499: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:48.064067 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:56:48.066218 26947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbb45df842f346ebb62f8b342f293499: No bootstrap required, opened a new log
I20260808 03:56:48.075592 26952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:48.069329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52425 (local address 127.26.146.190:43647)
0808 03:56:48.069498 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:48.069504 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:48.069820 (+ 316us) server_negotiation.cc:407] Connection header received
0808 03:56:48.069972 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:48.069978 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:48.070048 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:48.070156 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:48.070594 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:48.071740 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:48.073086 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:48.073766 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:48.073905 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:48.074857 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:48.074863 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:48.074868 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:48.074886 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:48.074946 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:48.074967 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:48.075076 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:48.075211 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 03:56:48.075261 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 03:56:48.075598 26947 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbb45df842f346ebb62f8b342f293499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb45df842f346ebb62f8b342f293499" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 43647 } }
I20260808 03:56:48.076100 26947 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbb45df842f346ebb62f8b342f293499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:48.076222 26947 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbb45df842f346ebb62f8b342f293499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbb45df842f346ebb62f8b342f293499, State: Initialized, Role: FOLLOWER
I20260808 03:56:48.076979 26947 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbb45df842f346ebb62f8b342f293499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb45df842f346ebb62f8b342f293499" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 43647 } }
I20260808 03:56:48.077270 26947 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbb45df842f346ebb62f8b342f293499 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:56:48.077414 26947 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbb45df842f346ebb62f8b342f293499 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:56:48.077567 26947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbb45df842f346ebb62f8b342f293499 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:48.079640 26947 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbb45df842f346ebb62f8b342f293499 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb45df842f346ebb62f8b342f293499" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 43647 } }
I20260808 03:56:48.080132 26947 leader_election.cc:302] T 00000000000000000000000000000000 P dbb45df842f346ebb62f8b342f293499 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbb45df842f346ebb62f8b342f293499; no voters:
I20260808 03:56:48.080673 26947 leader_election.cc:288] T 00000000000000000000000000000000 P dbb45df842f346ebb62f8b342f293499 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:56:48.080782 26962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbb45df842f346ebb62f8b342f293499 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:48.081923 26962 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbb45df842f346ebb62f8b342f293499 [term 1 LEADER]: Becoming Leader. State: Replica: dbb45df842f346ebb62f8b342f293499, State: Running, Role: LEADER
I20260808 03:56:48.082412 26962 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbb45df842f346ebb62f8b342f293499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb45df842f346ebb62f8b342f293499" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 43647 } }
I20260808 03:56:48.082715 26947 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbb45df842f346ebb62f8b342f293499 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:48.085152 26963 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbb45df842f346ebb62f8b342f293499 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbb45df842f346ebb62f8b342f293499" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb45df842f346ebb62f8b342f293499" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 43647 } } }
I20260808 03:56:48.085132 26964 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbb45df842f346ebb62f8b342f293499 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbb45df842f346ebb62f8b342f293499. Latest consensus state: current_term: 1 leader_uuid: "dbb45df842f346ebb62f8b342f293499" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb45df842f346ebb62f8b342f293499" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 43647 } } }
I20260808 03:56:48.085563 26963 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbb45df842f346ebb62f8b342f293499 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:48.085610 26964 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbb45df842f346ebb62f8b342f293499 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:48.086632 26973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:56:48.092252 26973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:56:48.100601 26973 catalog_manager.cc:1347] Generated new cluster ID: 35525d0f53a444e2b13a37dc65e9a5c0
I20260808 03:56:48.100703 26973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:56:48.119504 26973 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:56:48.121544 26973 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:56:48.131017 26973 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbb45df842f346ebb62f8b342f293499: Generated new TSK 0
I20260808 03:56:48.131877 26973 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:56:48.143249 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 26863
W20260808 03:56:48.144407 26941 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34709
W20260808 03:56:48.145187 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 26863: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:56:48.145331 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 26863
W20260808 03:56:48.171553 26843 connection.cc:535] client connection to 127.26.146.190:43647 recv error: Network error: failed to read from TLS socket (remote: 127.26.146.190:43647): Connection reset by peer (error 104)
2026-08-08T03:56:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2078 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:56:48 dist-test-slave-jlzn krb5kdc[26988](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:56:48 dist-test-slave-jlzn krb5kdc[26988](info): set up 2 sockets
Aug 08 03:56:48 dist-test-slave-jlzn krb5kdc[26988](info): commencing operation
krb5kdc: starting...
W20260808 03:56:49.364879 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:56:49 dist-test-slave-jlzn krb5kdc[26988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:56:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:56:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:56:49.679330 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:43009
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:46335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:43009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:50.022886 27004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:50.023172 27004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:50.023272 27004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:50.030412 27004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:50.030543 27004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:50.030606 27004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:50.030668 27004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:50.030714 27004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:50.040565 27004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:46335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:43009
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:43009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:50.041952 27004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:50.043289 27004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:50.054723 27011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:50.054870 27013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:50.054870 27015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:50.055893 27004 server_base.cc:1034] running on GCE node
I20260808 03:56:50.056686 27004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:50.058054 27004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:50.059286 27004 hybrid_clock.cc:648] HybridClock initialized: now 1786161410059238 us; error 56 us; skew 500 ppm
I20260808 03:56:50.059646 27004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:50 dist-test-slave-jlzn krb5kdc[26988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161410, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:50.062835 27004 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:56:50.064327 27004 webserver.cc:460] Webserver started at http://127.26.146.190:40323/ using document root <none> and password file <none>
I20260808 03:56:50.064985 27004 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:50.065124 27004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:50.065414 27004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:50.068392 27004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "9abbd651f5094ce789ff517bcc2fff04"
format_stamp: "Formatted at 2026-08-08 03:56:50 on dist-test-slave-jlzn"
server_key: "5579d8258eda2ef394dba99993804b03"
server_key_iv: "508205ba336edfb940779e782a79ba82"
server_key_version: "encryptionkey@0"
I20260808 03:56:50.069164 27004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "9abbd651f5094ce789ff517bcc2fff04"
format_stamp: "Formatted at 2026-08-08 03:56:50 on dist-test-slave-jlzn"
server_key: "5579d8258eda2ef394dba99993804b03"
server_key_iv: "508205ba336edfb940779e782a79ba82"
server_key_version: "encryptionkey@0"
I20260808 03:56:50.074337 27004 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260808 03:56:50.077750 27024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:50.078994 27004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:56:50.079150 27004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "9abbd651f5094ce789ff517bcc2fff04"
format_stamp: "Formatted at 2026-08-08 03:56:50 on dist-test-slave-jlzn"
server_key: "5579d8258eda2ef394dba99993804b03"
server_key_iv: "508205ba336edfb940779e782a79ba82"
server_key_version: "encryptionkey@0"
I20260808 03:56:50.079324 27004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:50.084939 27004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:50.088888 27004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:50.089178 27004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:50.107069 27004 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:43009
I20260808 03:56:50.107142 27099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:43009 every 8 connection(s)
I20260808 03:56:50.108681 27004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:56:50.109768 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 27004
I20260808 03:56:50.109957 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:56:50.110512 27210 external_mini_cluster.cc:1428] Setting key 7f53f20fa4f004d9bef183b3b9aa6129
I20260808 03:56:50.113523 27100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:50.118180 27104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:50.112398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40223 (local address 127.26.146.190:43009)
0808 03:56:50.113158 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:50.113170 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:50.113210 (+ 40us) server_negotiation.cc:407] Connection header received
0808 03:56:50.114889 (+ 1679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:50.114913 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:50.115226 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:50.115555 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:50.116620 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:50.116626 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:50.116638 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:50.116652 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:50.116727 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:50.116747 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:50.116963 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:50.117099 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 03:56:50.117276 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":312,"threads_started":1}
I20260808 03:56:50.121764 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:56:50.124689 27100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9abbd651f5094ce789ff517bcc2fff04: Bootstrap starting.
I20260808 03:56:50.128257 27100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9abbd651f5094ce789ff517bcc2fff04: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:50.128365 27104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:50.126106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37773 (local address 127.26.146.190:43009)
0808 03:56:50.126287 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:50.126293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:50.126513 (+ 220us) server_negotiation.cc:407] Connection header received
0808 03:56:50.126659 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:50.126664 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:50.126723 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:50.126829 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:50.127714 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:50.127719 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:50.127723 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:50.127741 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:50.127788 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:50.127827 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:50.127944 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:50.128022 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 03:56:50.128079 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 03:56:50.129520 27100 log.cc:824] T 00000000000000000000000000000000 P 9abbd651f5094ce789ff517bcc2fff04: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:50.132314 27100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9abbd651f5094ce789ff517bcc2fff04: No bootstrap required, opened a new log
I20260808 03:56:50.139876 27100 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9abbd651f5094ce789ff517bcc2fff04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9abbd651f5094ce789ff517bcc2fff04" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 43009 } }
I20260808 03:56:50.140239 27100 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9abbd651f5094ce789ff517bcc2fff04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:50.140337 27100 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9abbd651f5094ce789ff517bcc2fff04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9abbd651f5094ce789ff517bcc2fff04, State: Initialized, Role: FOLLOWER
I20260808 03:56:50.140930 27100 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9abbd651f5094ce789ff517bcc2fff04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9abbd651f5094ce789ff517bcc2fff04" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 43009 } }
I20260808 03:56:50.141150 27100 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9abbd651f5094ce789ff517bcc2fff04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:56:50.141289 27100 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9abbd651f5094ce789ff517bcc2fff04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:56:50.141443 27100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9abbd651f5094ce789ff517bcc2fff04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:50.143618 27100 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9abbd651f5094ce789ff517bcc2fff04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9abbd651f5094ce789ff517bcc2fff04" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 43009 } }
I20260808 03:56:50.144111 27100 leader_election.cc:302] T 00000000000000000000000000000000 P 9abbd651f5094ce789ff517bcc2fff04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9abbd651f5094ce789ff517bcc2fff04; no voters:
I20260808 03:56:50.144744 27100 leader_election.cc:288] T 00000000000000000000000000000000 P 9abbd651f5094ce789ff517bcc2fff04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:56:50.144888 27114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9abbd651f5094ce789ff517bcc2fff04 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:50.145632 27114 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9abbd651f5094ce789ff517bcc2fff04 [term 1 LEADER]: Becoming Leader. State: Replica: 9abbd651f5094ce789ff517bcc2fff04, State: Running, Role: LEADER
I20260808 03:56:50.146168 27114 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9abbd651f5094ce789ff517bcc2fff04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9abbd651f5094ce789ff517bcc2fff04" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 43009 } }
I20260808 03:56:50.146721 27100 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9abbd651f5094ce789ff517bcc2fff04 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:50.148948 27116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9abbd651f5094ce789ff517bcc2fff04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9abbd651f5094ce789ff517bcc2fff04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9abbd651f5094ce789ff517bcc2fff04" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 43009 } } }
I20260808 03:56:50.148974 27118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9abbd651f5094ce789ff517bcc2fff04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9abbd651f5094ce789ff517bcc2fff04. Latest consensus state: current_term: 1 leader_uuid: "9abbd651f5094ce789ff517bcc2fff04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9abbd651f5094ce789ff517bcc2fff04" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 43009 } } }
I20260808 03:56:50.149379 27116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9abbd651f5094ce789ff517bcc2fff04 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:50.149379 27118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9abbd651f5094ce789ff517bcc2fff04 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:50.153272 27125 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:56:50.158320 27125 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:56:50.167177 27125 catalog_manager.cc:1347] Generated new cluster ID: 609231b84ccf46c48d305e27469e0501
I20260808 03:56:50.167287 27125 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:56:50.193511 27125 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:56:50.193933 27125 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:56:50.207057 27125 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9abbd651f5094ce789ff517bcc2fff04: Generated new TSK 0
I20260808 03:56:50.207921 27125 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:56:50.218802 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 27004
W20260808 03:56:50.219955 27090 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40223
W20260808 03:56:50.220695 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 27004: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:56:50.220841 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 27004
2026-08-08T03:56:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2076 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:56:50 dist-test-slave-jlzn krb5kdc[27151](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:56:50 dist-test-slave-jlzn krb5kdc[27151](info): set up 2 sockets
Aug 08 03:56:50 dist-test-slave-jlzn krb5kdc[27151](info): commencing operation
krb5kdc: starting...
W20260808 03:56:51.441483 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.002s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:56:51 dist-test-slave-jlzn krb5kdc[27151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:56:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:56:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:56:51.759697 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:39345
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:46387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:39345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:52.104009 27167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:52.104291 27167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:52.104423 27167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:52.111614 27167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:52.111724 27167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:52.111784 27167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:52.111882 27167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:52.111937 27167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:52.121726 27167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:46387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:39345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:39345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:52.123025 27167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:52.124361 27167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:52.135900 27178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:52.135944 27179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:52.135912 27182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:52.136713 27167 server_base.cc:1034] running on GCE node
I20260808 03:56:52.137428 27167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:52.138756 27167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:52.139940 27167 hybrid_clock.cc:648] HybridClock initialized: now 1786161412139903 us; error 28 us; skew 500 ppm
I20260808 03:56:52.140275 27167 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:52 dist-test-slave-jlzn krb5kdc[27151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161412, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:52.146574 27167 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:56:52.148089 27167 webserver.cc:460] Webserver started at http://127.26.146.190:36747/ using document root <none> and password file <none>
I20260808 03:56:52.148751 27167 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:52.148873 27167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:52.149159 27167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:52.152154 27167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "a2bba86afc4d4e1ea1b260926ac8b53e"
format_stamp: "Formatted at 2026-08-08 03:56:52 on dist-test-slave-jlzn"
server_key: "16ae4f88f86e33e6f2ef6fd43cb0e318"
server_key_iv: "84343e51e5d55abce0f825fc2d295011"
server_key_version: "encryptionkey@0"
I20260808 03:56:52.152900 27167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "a2bba86afc4d4e1ea1b260926ac8b53e"
format_stamp: "Formatted at 2026-08-08 03:56:52 on dist-test-slave-jlzn"
server_key: "16ae4f88f86e33e6f2ef6fd43cb0e318"
server_key_iv: "84343e51e5d55abce0f825fc2d295011"
server_key_version: "encryptionkey@0"
I20260808 03:56:52.157953 27167 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:56:52.161393 27191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:52.162688 27167 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260808 03:56:52.162847 27167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "a2bba86afc4d4e1ea1b260926ac8b53e"
format_stamp: "Formatted at 2026-08-08 03:56:52 on dist-test-slave-jlzn"
server_key: "16ae4f88f86e33e6f2ef6fd43cb0e318"
server_key_iv: "84343e51e5d55abce0f825fc2d295011"
server_key_version: "encryptionkey@0"
I20260808 03:56:52.163036 27167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:52.180709 27167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:52.184836 27167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:52.185104 27167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:52.203646 27167 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:39345
I20260808 03:56:52.203678 27264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:39345 every 8 connection(s)
I20260808 03:56:52.205098 27167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:56:52.208946 27265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:52.212121 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 27167
I20260808 03:56:52.212308 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:56:52.212845 27210 external_mini_cluster.cc:1428] Setting key 3c8465a2d24419ccd8c545fe169ac932
I20260808 03:56:52.221297 27265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2bba86afc4d4e1ea1b260926ac8b53e: Bootstrap starting.
I20260808 03:56:52.225023 27265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2bba86afc4d4e1ea1b260926ac8b53e: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:52.226343 27265 log.cc:824] T 00000000000000000000000000000000 P a2bba86afc4d4e1ea1b260926ac8b53e: Log is configured to *not* fsync() on all Append() calls
Aug 08 03:56:52 dist-test-slave-jlzn krb5kdc[27151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:56:52.229683 27265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2bba86afc4d4e1ea1b260926ac8b53e: No bootstrap required, opened a new log
I20260808 03:56:52.236161 27269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:52.214780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58573 (local address 127.26.146.190:39345)
0808 03:56:52.215550 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:52.215562 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:52.215612 (+ 50us) server_negotiation.cc:407] Connection header received
0808 03:56:52.217328 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:52.217350 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:52.217749 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:52.218067 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:52.219834 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:52.221142 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:52.222865 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:52.223549 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:52.223680 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:52.228925 (+ 5245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:52.228933 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:52.228949 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:52.228988 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:52.232125 (+ 3137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:52.232893 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:52.232899 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:52.232905 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:52.233075 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:52.233473 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:52.233477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:52.233479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:52.234009 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:52.234229 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:52.234715 (+ 486us) server_negotiation.cc:299] Negotiation successful
0808 03:56:52.235076 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":364,"threads_started":1}
I20260808 03:56:52.238758 27265 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2bba86afc4d4e1ea1b260926ac8b53e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2bba86afc4d4e1ea1b260926ac8b53e" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 39345 } }
I20260808 03:56:52.239184 27265 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2bba86afc4d4e1ea1b260926ac8b53e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:52.239301 27265 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2bba86afc4d4e1ea1b260926ac8b53e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2bba86afc4d4e1ea1b260926ac8b53e, State: Initialized, Role: FOLLOWER
I20260808 03:56:52.239962 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:56:52.239953 27265 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2bba86afc4d4e1ea1b260926ac8b53e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2bba86afc4d4e1ea1b260926ac8b53e" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 39345 } }
I20260808 03:56:52.240217 27265 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2bba86afc4d4e1ea1b260926ac8b53e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:56:52.240343 27265 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2bba86afc4d4e1ea1b260926ac8b53e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:56:52.240490 27265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2bba86afc4d4e1ea1b260926ac8b53e [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:52.242669 27265 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2bba86afc4d4e1ea1b260926ac8b53e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2bba86afc4d4e1ea1b260926ac8b53e" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 39345 } }
I20260808 03:56:52.243244 27265 leader_election.cc:302] T 00000000000000000000000000000000 P a2bba86afc4d4e1ea1b260926ac8b53e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2bba86afc4d4e1ea1b260926ac8b53e; no voters:
I20260808 03:56:52.243856 27265 leader_election.cc:288] T 00000000000000000000000000000000 P a2bba86afc4d4e1ea1b260926ac8b53e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:56:52.244024 27275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2bba86afc4d4e1ea1b260926ac8b53e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:52.249406 27275 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2bba86afc4d4e1ea1b260926ac8b53e [term 1 LEADER]: Becoming Leader. State: Replica: a2bba86afc4d4e1ea1b260926ac8b53e, State: Running, Role: LEADER
I20260808 03:56:52.250072 27275 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2bba86afc4d4e1ea1b260926ac8b53e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2bba86afc4d4e1ea1b260926ac8b53e" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 39345 } }
I20260808 03:56:52.250507 27265 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2bba86afc4d4e1ea1b260926ac8b53e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:52.252866 27278 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2bba86afc4d4e1ea1b260926ac8b53e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2bba86afc4d4e1ea1b260926ac8b53e. Latest consensus state: current_term: 1 leader_uuid: "a2bba86afc4d4e1ea1b260926ac8b53e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2bba86afc4d4e1ea1b260926ac8b53e" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 39345 } } }
I20260808 03:56:52.253273 27278 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2bba86afc4d4e1ea1b260926ac8b53e [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:52.253841 27277 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2bba86afc4d4e1ea1b260926ac8b53e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2bba86afc4d4e1ea1b260926ac8b53e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2bba86afc4d4e1ea1b260926ac8b53e" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 39345 } } }
I20260808 03:56:52.254138 27277 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2bba86afc4d4e1ea1b260926ac8b53e [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:52.264586 27269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:52.244384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52813 (local address 127.26.146.190:39345)
0808 03:56:52.244569 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:52.244576 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:52.248357 (+ 3781us) server_negotiation.cc:407] Connection header received
0808 03:56:52.248526 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:52.248531 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:52.248620 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:52.248725 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:52.250576 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:52.251338 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:52.253071 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:52.253760 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:52.253882 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:52.257374 (+ 3492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:52.257383 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:52.257388 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:52.257430 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:52.261100 (+ 3670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:52.261954 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:52.261960 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:52.261963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:52.262129 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:52.262609 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:52.262614 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:52.262618 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:52.263060 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:52.263309 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:52.263940 (+ 631us) server_negotiation.cc:299] Negotiation successful
0808 03:56:52.264196 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 03:56:52.274219 27289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:56:52.280370 27289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:56:52.288574 27289 catalog_manager.cc:1347] Generated new cluster ID: e9c214d1b05e42b099bb0ea2bcf00890
I20260808 03:56:52.288691 27289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:56:52.311900 27289 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:56:52.313817 27289 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:56:52.322660 27289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2bba86afc4d4e1ea1b260926ac8b53e: Generated new TSK 0
I20260808 03:56:52.323650 27289 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:56:52.338075 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 27167
I20260808 03:56:52.339120 27259 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:56:52.431365 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 27167
2026-08-08T03:56:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2204 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:56:52 dist-test-slave-jlzn krb5kdc[27318](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:56:52 dist-test-slave-jlzn krb5kdc[27318](info): set up 2 sockets
Aug 08 03:56:52 dist-test-slave-jlzn krb5kdc[27318](info): commencing operation
krb5kdc: starting...
W20260808 03:56:53.646549 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.001s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:56:53 dist-test-slave-jlzn krb5kdc[27318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:56:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:56:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:56:53.965355 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:40203
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:40203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:54.298712 27335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:54.298993 27335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:54.299119 27335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:54.306058 27335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:54.306169 27335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:54.306239 27335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:54.306318 27335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:54.306378 27335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:54.315968 27335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:40203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:40203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:54.317272 27335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:54.318774 27335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:54.330227 27340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:54.330271 27343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:54.330188 27347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:54.333242 27335 server_base.cc:1034] running on GCE node
I20260808 03:56:54.334029 27335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:54.335392 27335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:54.336572 27335 hybrid_clock.cc:648] HybridClock initialized: now 1786161414336521 us; error 46 us; skew 500 ppm
I20260808 03:56:54.336944 27335 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:54 dist-test-slave-jlzn krb5kdc[27318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161414, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:54.340224 27335 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:56:54.341639 27335 webserver.cc:460] Webserver started at http://127.26.146.190:37517/ using document root <none> and password file <none>
I20260808 03:56:54.342289 27335 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:54.342388 27335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:54.342653 27335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:54.345624 27335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "bffde5681c8247f4acf77042c2489d42"
format_stamp: "Formatted at 2026-08-08 03:56:54 on dist-test-slave-jlzn"
server_key: "7a4c3bef29dd609d603a21a1ddf03815"
server_key_iv: "f4bff514ae5b1d0b786d5b41b055a523"
server_key_version: "encryptionkey@0"
I20260808 03:56:54.346329 27335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "bffde5681c8247f4acf77042c2489d42"
format_stamp: "Formatted at 2026-08-08 03:56:54 on dist-test-slave-jlzn"
server_key: "7a4c3bef29dd609d603a21a1ddf03815"
server_key_iv: "f4bff514ae5b1d0b786d5b41b055a523"
server_key_version: "encryptionkey@0"
I20260808 03:56:54.351244 27335 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 03:56:54.354547 27358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:54.355769 27335 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:56:54.355964 27335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "bffde5681c8247f4acf77042c2489d42"
format_stamp: "Formatted at 2026-08-08 03:56:54 on dist-test-slave-jlzn"
server_key: "7a4c3bef29dd609d603a21a1ddf03815"
server_key_iv: "f4bff514ae5b1d0b786d5b41b055a523"
server_key_version: "encryptionkey@0"
I20260808 03:56:54.356135 27335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:54.386270 27335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:54.390763 27335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:54.391004 27335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:54.408409 27335 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:40203
I20260808 03:56:54.408435 27425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:40203 every 8 connection(s)
I20260808 03:56:54.409754 27335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:56:54.413594 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:54.415349 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 27335
I20260808 03:56:54.415524 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:56:54.416131 27210 external_mini_cluster.cc:1428] Setting key 506611c503f74ab74a100b8bf7da123f
I20260808 03:56:54.427546 27426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bffde5681c8247f4acf77042c2489d42: Bootstrap starting.
I20260808 03:56:54.429476 27431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:54.418008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44841 (local address 127.26.146.190:40203)
0808 03:56:54.418881 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:54.418892 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:54.418937 (+ 45us) server_negotiation.cc:407] Connection header received
0808 03:56:54.420624 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:54.420652 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:54.421042 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:54.421359 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:54.422015 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:54.423292 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:54.425124 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:54.425829 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:54.425951 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:54.427273 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:54.427281 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:54.427294 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:54.427867 (+ 573us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:54.427946 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:54.427976 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:54.428066 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:54.428264 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 03:56:54.428490 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":377,"threads_started":1}
I20260808 03:56:54.430961 27426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bffde5681c8247f4acf77042c2489d42: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:54.432286 27426 log.cc:824] T 00000000000000000000000000000000 P bffde5681c8247f4acf77042c2489d42: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:54.433364 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:56:54.435081 27426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bffde5681c8247f4acf77042c2489d42: No bootstrap required, opened a new log
I20260808 03:56:54.444442 27431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:54.437681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50407 (local address 127.26.146.190:40203)
0808 03:56:54.437856 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:54.437862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:54.438171 (+ 309us) server_negotiation.cc:407] Connection header received
0808 03:56:54.438321 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:54.438328 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:54.438409 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:54.438536 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:54.439035 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:54.440285 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:54.441707 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:54.442516 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:54.442646 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:54.443628 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:54.443635 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:54.443639 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:54.443660 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:54.443706 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:54.443725 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:54.443872 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:54.444017 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 03:56:54.444110 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 03:56:54.444260 27426 raft_consensus.cc:348] T 00000000000000000000000000000000 P bffde5681c8247f4acf77042c2489d42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bffde5681c8247f4acf77042c2489d42" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 40203 } }
I20260808 03:56:54.444797 27426 raft_consensus.cc:374] T 00000000000000000000000000000000 P bffde5681c8247f4acf77042c2489d42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:54.444911 27426 raft_consensus.cc:729] T 00000000000000000000000000000000 P bffde5681c8247f4acf77042c2489d42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bffde5681c8247f4acf77042c2489d42, State: Initialized, Role: FOLLOWER
I20260808 03:56:54.445534 27426 consensus_queue.cc:260] T 00000000000000000000000000000000 P bffde5681c8247f4acf77042c2489d42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bffde5681c8247f4acf77042c2489d42" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 40203 } }
I20260808 03:56:54.445768 27426 raft_consensus.cc:388] T 00000000000000000000000000000000 P bffde5681c8247f4acf77042c2489d42 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:56:54.445912 27426 raft_consensus.cc:482] T 00000000000000000000000000000000 P bffde5681c8247f4acf77042c2489d42 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:56:54.446079 27426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bffde5681c8247f4acf77042c2489d42 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:54.448298 27426 raft_consensus.cc:504] T 00000000000000000000000000000000 P bffde5681c8247f4acf77042c2489d42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bffde5681c8247f4acf77042c2489d42" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 40203 } }
I20260808 03:56:54.448765 27426 leader_election.cc:302] T 00000000000000000000000000000000 P bffde5681c8247f4acf77042c2489d42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bffde5681c8247f4acf77042c2489d42; no voters:
I20260808 03:56:54.449368 27426 leader_election.cc:288] T 00000000000000000000000000000000 P bffde5681c8247f4acf77042c2489d42 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:56:54.449474 27442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bffde5681c8247f4acf77042c2489d42 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:54.450132 27442 raft_consensus.cc:686] T 00000000000000000000000000000000 P bffde5681c8247f4acf77042c2489d42 [term 1 LEADER]: Becoming Leader. State: Replica: bffde5681c8247f4acf77042c2489d42, State: Running, Role: LEADER
I20260808 03:56:54.450659 27442 consensus_queue.cc:237] T 00000000000000000000000000000000 P bffde5681c8247f4acf77042c2489d42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bffde5681c8247f4acf77042c2489d42" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 40203 } }
I20260808 03:56:54.451608 27426 sys_catalog.cc:564] T 00000000000000000000000000000000 P bffde5681c8247f4acf77042c2489d42 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:54.453961 27444 sys_catalog.cc:455] T 00000000000000000000000000000000 P bffde5681c8247f4acf77042c2489d42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bffde5681c8247f4acf77042c2489d42. Latest consensus state: current_term: 1 leader_uuid: "bffde5681c8247f4acf77042c2489d42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bffde5681c8247f4acf77042c2489d42" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 40203 } } }
I20260808 03:56:54.454012 27443 sys_catalog.cc:455] T 00000000000000000000000000000000 P bffde5681c8247f4acf77042c2489d42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bffde5681c8247f4acf77042c2489d42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bffde5681c8247f4acf77042c2489d42" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 40203 } } }
I20260808 03:56:54.454372 27444 sys_catalog.cc:458] T 00000000000000000000000000000000 P bffde5681c8247f4acf77042c2489d42 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:54.454377 27443 sys_catalog.cc:458] T 00000000000000000000000000000000 P bffde5681c8247f4acf77042c2489d42 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:54.459491 27455 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:56:54.465636 27455 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:56:54.475512 27455 catalog_manager.cc:1347] Generated new cluster ID: 579067fc100e458bb34b503d7c0879e0
I20260808 03:56:54.475728 27455 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:56:54.484309 27455 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:56:54.485620 27455 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:56:54.493355 27455 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bffde5681c8247f4acf77042c2489d42: Generated new TSK 0
I20260808 03:56:54.494236 27455 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:56:54.500487 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 27335
W20260808 03:56:54.501560 27420 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44841
W20260808 03:56:54.502240 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 27335: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:56:54.502398 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 27335
2026-08-08T03:56:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2074 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:56:54 dist-test-slave-jlzn krb5kdc[27477](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:56:54 dist-test-slave-jlzn krb5kdc[27477](info): set up 2 sockets
Aug 08 03:56:54 dist-test-slave-jlzn krb5kdc[27477](info): commencing operation
krb5kdc: starting...
W20260808 03:56:55.730563 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.004s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:56:55 dist-test-slave-jlzn krb5kdc[27477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:56:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:56:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:56:56.051590 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:34055
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:33743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:34055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:56.387681 27497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:56.388012 27497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:56.388160 27497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:56.395216 27497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:56.395327 27497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:56.395383 27497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:56.395469 27497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:56.395529 27497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:56.405272 27497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:33743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:34055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:34055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:56.406598 27497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:56.407950 27497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:56.419315 27504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:56.419268 27505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:56.419257 27508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:56.420228 27497 server_base.cc:1034] running on GCE node
I20260808 03:56:56.420913 27497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:56.422181 27497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:56.423377 27497 hybrid_clock.cc:648] HybridClock initialized: now 1786161416423341 us; error 27 us; skew 500 ppm
I20260808 03:56:56.423702 27497 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:56 dist-test-slave-jlzn krb5kdc[27477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161416, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:56.426862 27497 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:56:56.428300 27497 webserver.cc:460] Webserver started at http://127.26.146.190:45671/ using document root <none> and password file <none>
I20260808 03:56:56.428946 27497 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:56.429059 27497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:56.429353 27497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:56.432323 27497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "a2eb367011f7483a8381372087b79f8d"
format_stamp: "Formatted at 2026-08-08 03:56:56 on dist-test-slave-jlzn"
server_key: "2467fcea504ad09c74c0c95ca8c6525d"
server_key_iv: "98daee43d2fa8bc160627a69c5a3771b"
server_key_version: "encryptionkey@0"
I20260808 03:56:56.433038 27497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "a2eb367011f7483a8381372087b79f8d"
format_stamp: "Formatted at 2026-08-08 03:56:56 on dist-test-slave-jlzn"
server_key: "2467fcea504ad09c74c0c95ca8c6525d"
server_key_iv: "98daee43d2fa8bc160627a69c5a3771b"
server_key_version: "encryptionkey@0"
I20260808 03:56:56.438079 27497 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:56:56.441562 27517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:56.442827 27497 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 03:56:56.442988 27497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "a2eb367011f7483a8381372087b79f8d"
format_stamp: "Formatted at 2026-08-08 03:56:56 on dist-test-slave-jlzn"
server_key: "2467fcea504ad09c74c0c95ca8c6525d"
server_key_iv: "98daee43d2fa8bc160627a69c5a3771b"
server_key_version: "encryptionkey@0"
I20260808 03:56:56.443176 27497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:56.448696 27497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:56.452620 27497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:56.452881 27497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:56.469959 27497 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:34055
I20260808 03:56:56.469991 27592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:34055 every 8 connection(s)
I20260808 03:56:56.471364 27497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:56:56.475273 27593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:56.480893 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 27497
I20260808 03:56:56.481062 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:56:56.481462 27210 external_mini_cluster.cc:1428] Setting key 0e4dd6c07a60fab65eeae37682ec7877
I20260808 03:56:56.486265 27593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2eb367011f7483a8381372087b79f8d: Bootstrap starting.
I20260808 03:56:56.489076 27596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:56.483224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58277 (local address 127.26.146.190:34055)
0808 03:56:56.484074 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:56.484086 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:56.484131 (+ 45us) server_negotiation.cc:407] Connection header received
0808 03:56:56.485787 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:56.485814 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:56.486127 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:56.486486 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:56.487542 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:56.487549 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:56.487562 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:56.487576 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:56.487642 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:56.487666 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:56.487905 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:56.488049 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 03:56:56.488233 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":375,"threads_started":1}
I20260808 03:56:56.489909 27593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2eb367011f7483a8381372087b79f8d: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:56.491087 27593 log.cc:824] T 00000000000000000000000000000000 P a2eb367011f7483a8381372087b79f8d: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:56.492965 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:56:56.494112 27593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2eb367011f7483a8381372087b79f8d: No bootstrap required, opened a new log
I20260808 03:56:56.499403 27596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:56.497231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36431 (local address 127.26.146.190:34055)
0808 03:56:56.497409 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:56.497415 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:56.497714 (+ 299us) server_negotiation.cc:407] Connection header received
0808 03:56:56.497836 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:56.497841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:56.497896 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:56.498006 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:56.498793 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:56.498803 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:56.498808 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:56:56.498826 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:56.498873 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:56:56.498894 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:56.499022 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:56.499065 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 03:56:56.499123 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 03:56:56.502337 27593 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2eb367011f7483a8381372087b79f8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2eb367011f7483a8381372087b79f8d" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 34055 } }
I20260808 03:56:56.502671 27593 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2eb367011f7483a8381372087b79f8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:56:56.502744 27593 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2eb367011f7483a8381372087b79f8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2eb367011f7483a8381372087b79f8d, State: Initialized, Role: FOLLOWER
I20260808 03:56:56.503221 27593 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2eb367011f7483a8381372087b79f8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2eb367011f7483a8381372087b79f8d" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 34055 } }
I20260808 03:56:56.503427 27593 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2eb367011f7483a8381372087b79f8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:56:56.503527 27593 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2eb367011f7483a8381372087b79f8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:56:56.503660 27593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2eb367011f7483a8381372087b79f8d [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:56.505784 27593 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2eb367011f7483a8381372087b79f8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2eb367011f7483a8381372087b79f8d" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 34055 } }
I20260808 03:56:56.506240 27593 leader_election.cc:302] T 00000000000000000000000000000000 P a2eb367011f7483a8381372087b79f8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2eb367011f7483a8381372087b79f8d; no voters:
I20260808 03:56:56.506832 27593 leader_election.cc:288] T 00000000000000000000000000000000 P a2eb367011f7483a8381372087b79f8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:56:56.506933 27606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2eb367011f7483a8381372087b79f8d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:56.507607 27606 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2eb367011f7483a8381372087b79f8d [term 1 LEADER]: Becoming Leader. State: Replica: a2eb367011f7483a8381372087b79f8d, State: Running, Role: LEADER
I20260808 03:56:56.508215 27606 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2eb367011f7483a8381372087b79f8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2eb367011f7483a8381372087b79f8d" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 34055 } }
I20260808 03:56:56.508972 27593 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2eb367011f7483a8381372087b79f8d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:56.511050 27610 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2eb367011f7483a8381372087b79f8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2eb367011f7483a8381372087b79f8d. Latest consensus state: current_term: 1 leader_uuid: "a2eb367011f7483a8381372087b79f8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2eb367011f7483a8381372087b79f8d" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 34055 } } }
I20260808 03:56:56.511471 27610 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2eb367011f7483a8381372087b79f8d [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:56.511530 27607 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2eb367011f7483a8381372087b79f8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2eb367011f7483a8381372087b79f8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2eb367011f7483a8381372087b79f8d" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 34055 } } }
I20260808 03:56:56.511900 27607 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2eb367011f7483a8381372087b79f8d [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:56.514276 27618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:56:56.519196 27618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:56:56.530333 27618 catalog_manager.cc:1347] Generated new cluster ID: 59197659b56f42eebe20f27a55341a2e
I20260808 03:56:56.530447 27618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:56:56.541875 27618 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:56:56.542291 27618 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:56:56.550370 27618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2eb367011f7483a8381372087b79f8d: Generated new TSK 0
I20260808 03:56:56.551146 27618 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:56:56.558482 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 27497
W20260808 03:56:56.560493 27584 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58277
W20260808 03:56:56.561260 27210 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 27497: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:56:56.561405 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 27497
2026-08-08T03:56:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2058 ms)
[----------] 18 tests from AuthTokenIssuingTest (37809 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:56:56 dist-test-slave-jlzn krb5kdc[27639](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:56:56 dist-test-slave-jlzn krb5kdc[27639](info): set up 2 sockets
Aug 08 03:56:56 dist-test-slave-jlzn krb5kdc[27639](info): commencing operation
krb5kdc: starting...
W20260808 03:56:57.795382 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.003s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:56:58 dist-test-slave-jlzn krb5kdc[27639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:56:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:56:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:56:58.115959 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:36417
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:36417,127.26.146.189:33477,127.26.146.188:40515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:56:58.452608 27655 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 03:56:58.452926 27655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:58.452999 27655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:58.453088 27655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:58.460119 27655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:58.460232 27655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:58.460299 27655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:58.460355 27655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:58.460419 27655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:58.470152 27655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:36417,127.26.146.189:33477,127.26.146.188:40515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:36417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:58.471406 27655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:58.472710 27655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:58.484251 27662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:58.484323 27663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:58.484436 27666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:58.485754 27655 server_base.cc:1034] running on GCE node
I20260808 03:56:58.486603 27655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:58.488055 27655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:58.489285 27655 hybrid_clock.cc:648] HybridClock initialized: now 1786161418489213 us; error 47 us; skew 500 ppm
I20260808 03:56:58.489604 27655 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:58 dist-test-slave-jlzn krb5kdc[27639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161418, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:58.496762 27655 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:56:58.498404 27655 webserver.cc:460] Webserver started at http://127.26.146.190:34145/ using document root <none> and password file <none>
I20260808 03:56:58.499128 27655 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:58.499234 27655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:58.499538 27655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:58.502604 27655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "57ada18c2ae14dbeaf97008eee195b04"
format_stamp: "Formatted at 2026-08-08 03:56:58 on dist-test-slave-jlzn"
server_key: "e9fe2b31186faaeab700ed4938d70a73"
server_key_iv: "ac18e2339c73aa6616c59d3059f059c7"
server_key_version: "encryptionkey@0"
I20260808 03:56:58.503405 27655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "57ada18c2ae14dbeaf97008eee195b04"
format_stamp: "Formatted at 2026-08-08 03:56:58 on dist-test-slave-jlzn"
server_key: "e9fe2b31186faaeab700ed4938d70a73"
server_key_iv: "ac18e2339c73aa6616c59d3059f059c7"
server_key_version: "encryptionkey@0"
I20260808 03:56:58.508692 27655 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 03:56:58.512257 27674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:58.513549 27655 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 03:56:58.513720 27655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "57ada18c2ae14dbeaf97008eee195b04"
format_stamp: "Formatted at 2026-08-08 03:56:58 on dist-test-slave-jlzn"
server_key: "e9fe2b31186faaeab700ed4938d70a73"
server_key_iv: "ac18e2339c73aa6616c59d3059f059c7"
server_key_version: "encryptionkey@0"
I20260808 03:56:58.513922 27655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:58.537084 27655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:58.541059 27655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:58.541317 27655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:58.558993 27655 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:36417
I20260808 03:56:58.559015 27752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:36417 every 8 connection(s)
I20260808 03:56:58.560359 27655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:56:58.564167 27753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:58.566071 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 27655
I20260808 03:56:58.566310 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:56:58.567164 27210 external_mini_cluster.cc:1428] Setting key c3d4011b324580c09d2ac76312fd2059
I20260808 03:56:58.573115 27753 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36417 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:56:58 dist-test-slave-jlzn krb5kdc[27639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161418, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:56:58.598747 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:58.577284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36417 (local address 127.0.0.1:56338)
0808 03:56:58.578252 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:58.578315 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:58.580846 (+ 2531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:58.581841 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:58.581852 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:58.582635 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:58.582986 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:58.583004 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:58.585219 (+ 2215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:58.585222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:58.586342 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:58.586359 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:58.587496 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:58.587501 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:58.587705 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:58.588372 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:56:58.588400 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:58.590817 (+ 2417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:58.594645 (+ 3828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:58.594655 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:58.594665 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:58.595059 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:58.595526 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:58.595529 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:58.595531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:58.595687 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:58.596606 (+ 919us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:58.596613 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:58.596802 (+ 189us) client_negotiation.cc:769] Sending connection context
0808 03:56:58.597032 (+ 230us) client_negotiation.cc:240] Negotiation successful
0808 03:56:58.597432 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"mutex_wait_us":36,"thread_start_us":399,"threads_started":1}
I20260808 03:56:58.598786 27758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:58.577284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56338 (local address 127.26.146.190:36417)
0808 03:56:58.579275 (+ 1991us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:58.579283 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:58.579330 (+ 47us) server_negotiation.cc:407] Connection header received
0808 03:56:58.581277 (+ 1947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:58.581283 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:58.581581 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:58.581771 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:58.583224 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:58.585056 (+ 1832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:58.586489 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:58.587273 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:58.587487 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:58.591105 (+ 3618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:58.591122 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:58.591140 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:58.591182 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:58.594515 (+ 3333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:58.595200 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:58.595208 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:58.595211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:58.595408 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:58.595793 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:58.595798 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:58.595833 (+ 35us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:58.596458 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:58.596643 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:58.597189 (+ 546us) server_negotiation.cc:299] Negotiation successful
0808 03:56:58.597654 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":138,"server-negotiator.queue_time_us":1784,"spinlock_wait_cycles":29952,"thread_start_us":1530,"threads_started":1}
I20260808 03:56:58.601868 27753 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.146.189@KRBTEST.COM; defaulting to no policy
I20260808 03:56:58.603096 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:58.602612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33477 (local address 127.0.0.1:45208)
0808 03:56:58.602800 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:58.602934 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260808 03:56:58.603367 27679 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33477: connect: Connection refused (error 111)
I20260808 03:56:58.604889 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:58.604540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33477 (local address 127.0.0.1:45212)
0808 03:56:58.604679 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:58.604750 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 03:56:58.605090 27753 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.189:33477: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33477: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.189.keytab.
Entry for principal kudu/127.26.146.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.189.keytab.
I20260808 03:56:58.643622 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.189:33477
--webserver_interface=127.26.146.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.189.keytab
--principal=kudu/127.26.146.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:36417,127.26.146.189:33477,127.26.146.188:40515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:56:58.654438 27753 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } attempt: 1
I20260808 03:56:58.655707 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:58.655214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33477 (local address 127.0.0.1:45228)
0808 03:56:58.655422 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:58.655564 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 03:56:58.656847 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:58.656431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33477 (local address 127.0.0.1:45234)
0808 03:56:58.656578 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:58.656663 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 03:56:58.657065 27753 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.189:33477: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33477: connect: Connection refused (error 111)
I20260808 03:56:58.725335 27753 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } attempt: 2
I20260808 03:56:58.726639 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:58.726054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33477 (local address 127.0.0.1:45244)
0808 03:56:58.726331 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:58.726446 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260808 03:56:58.727773 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:58.727331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33477 (local address 127.0.0.1:45256)
0808 03:56:58.727499 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:58.727595 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 03:56:58.728019 27753 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.189:33477: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33477: connect: Connection refused (error 111)
I20260808 03:56:58.819267 27753 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } attempt: 3
I20260808 03:56:58.820453 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:58.820011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33477 (local address 127.0.0.1:45264)
0808 03:56:58.820211 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:58.820307 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 03:56:58.821484 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:58.821136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33477 (local address 127.0.0.1:45272)
0808 03:56:58.821275 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:58.821348 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 03:56:58.821666 27753 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.189:33477: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33477: connect: Connection refused (error 111)
I20260808 03:56:58.964954 27753 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } attempt: 4
I20260808 03:56:58.966277 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:58.965720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33477 (local address 127.0.0.1:45278)
0808 03:56:58.966027 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:58.966131 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260808 03:56:58.967273 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:58.966884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33477 (local address 127.0.0.1:45292)
0808 03:56:58.967032 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:58.967146 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 03:56:58.967466 27753 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.189:33477: Network error: Client connection negotiation failed: client connection to 127.26.146.189:33477: connect: Connection refused (error 111)
W20260808 03:56:58.991791 27763 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 03:56:58.992183 27763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:58.992283 27763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:58.992398 27763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:56:58.999647 27763 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:58.999745 27763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:58.999867 27763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:58.999935 27763 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:59.000034 27763 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:59.009755 27763 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:36417,127.26.146.189:33477,127.26.146.188:40515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.189.keytab
--principal=kudu/127.26.146.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.189:33477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.146.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:59.011059 27763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:59.012440 27763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:59.023910 27770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:59.023857 27771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:59.023904 27773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:59.024777 27763 server_base.cc:1034] running on GCE node
I20260808 03:56:59.025496 27763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:59.026799 27763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:59.028012 27763 hybrid_clock.cc:648] HybridClock initialized: now 1786161419027950 us; error 35 us; skew 500 ppm
I20260808 03:56:59.028313 27763 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:59 dist-test-slave-jlzn krb5kdc[27639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161419, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:59.031311 27763 init.cc:377] Logged in from keytab as kudu/127.26.146.189@KRBTEST.COM (short username kudu)
I20260808 03:56:59.032699 27763 webserver.cc:460] Webserver started at http://127.26.146.189:42885/ using document root <none> and password file <none>
I20260808 03:56:59.033324 27763 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:59.033408 27763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:59.033674 27763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:59.036602 27763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-1/data/instance:
uuid: "07f7751a8d4e41ea84d3405260a8fe1e"
format_stamp: "Formatted at 2026-08-08 03:56:59 on dist-test-slave-jlzn"
server_key: "493fc263b97bf28ac808960a4199e8f9"
server_key_iv: "0117e8e15e456f57ea88cbe1ef82b902"
server_key_version: "encryptionkey@0"
I20260808 03:56:59.037324 27763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-1/wal/instance:
uuid: "07f7751a8d4e41ea84d3405260a8fe1e"
format_stamp: "Formatted at 2026-08-08 03:56:59 on dist-test-slave-jlzn"
server_key: "493fc263b97bf28ac808960a4199e8f9"
server_key_iv: "0117e8e15e456f57ea88cbe1ef82b902"
server_key_version: "encryptionkey@0"
I20260808 03:56:59.042164 27763 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 03:56:59.045495 27783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:59.046785 27763 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 03:56:59.046942 27763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-1/wal
uuid: "07f7751a8d4e41ea84d3405260a8fe1e"
format_stamp: "Formatted at 2026-08-08 03:56:59 on dist-test-slave-jlzn"
server_key: "493fc263b97bf28ac808960a4199e8f9"
server_key_iv: "0117e8e15e456f57ea88cbe1ef82b902"
server_key_version: "encryptionkey@0"
I20260808 03:56:59.047133 27763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:59.064929 27763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:59.068764 27763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:59.069015 27763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:59.086719 27763 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.189:33477
I20260808 03:56:59.086761 27847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.189:33477 every 8 connection(s)
I20260808 03:56:59.088054 27763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-1/data/info.pb
I20260808 03:56:59.091715 27849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:59.096470 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 27763
I20260808 03:56:59.096663 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-1/wal/instance
I20260808 03:56:59.097239 27210 external_mini_cluster.cc:1428] Setting key 6315e8499351d8a0e222bc206bb3c2d3
I20260808 03:56:59.098887 27849 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36417 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:56:59 dist-test-slave-jlzn krb5kdc[27639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161419, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.189@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:56:59.119663 27851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.102004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56348 (local address 127.26.146.190:36417)
0808 03:56:59.102516 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:59.102523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:59.103452 (+ 929us) server_negotiation.cc:407] Connection header received
0808 03:56:59.105351 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:59.105356 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:59.105432 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:59.105525 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:59.107034 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.107827 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.109276 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.109681 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.109789 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:59.113705 (+ 3916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:59.113712 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:59.113718 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:59.113767 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:59.116474 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.117335 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.117339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.117341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.117480 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.117931 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.117935 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.117937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.118235 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:59.118344 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:59.119216 (+ 872us) server_negotiation.cc:299] Negotiation successful
0808 03:56:59.119385 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":255,"threads_started":1}
I20260808 03:56:59.121008 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.102539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36417 (local address 127.0.0.1:56348)
0808 03:56:59.103274 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.103328 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:59.105092 (+ 1764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:59.105782 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:59.105790 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:59.106273 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:59.106831 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:59.106849 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.107974 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.107978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:59.109136 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:59.109148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.109894 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.109902 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:59.110120 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:59.110738 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:56:59.110760 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:59.113389 (+ 2629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:59.116633 (+ 3244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:59.116643 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:59.116662 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:59.117236 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:59.117588 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:59.117592 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:59.117596 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:59.117840 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:59.118365 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:59.118372 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:59.118824 (+ 452us) client_negotiation.cc:769] Sending connection context
0808 03:56:59.119106 (+ 282us) client_negotiation.cc:240] Negotiation successful
0808 03:56:59.119618 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":301,"threads_started":1}
I20260808 03:56:59.122772 27849 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.146.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.188@KRBTEST.COM" created.
Aug 08 03:56:59 dist-test-slave-jlzn krb5kdc[27639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161419, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.189@KRBTEST.COM for kudu/127.26.146.189@KRBTEST.COM
I20260808 03:56:59.140839 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.123400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33477 (local address 127.0.0.1:45296)
0808 03:56:59.123561 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.123580 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:59.123677 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:59.124676 (+ 999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:59.124686 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:59.125131 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:59.125302 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:59.125314 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.126719 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.126723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:59.128042 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:59.128058 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.128868 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.128873 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:59.129063 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:59.129702 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:56:59.129727 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:59.134640 (+ 4913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:59.138386 (+ 3746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:59.138390 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:59.138392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:59.138796 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:59.139173 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:59.139176 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:59.139178 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:59.139302 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:59.140028 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:59.140033 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:59.140165 (+ 132us) client_negotiation.cc:769] Sending connection context
0808 03:56:59.140285 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 03:56:59.140487 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 03:56:59.141103 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.123651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45296 (local address 127.26.146.189:33477)
0808 03:56:59.124254 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:59.124259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:59.124283 (+ 24us) server_negotiation.cc:407] Connection header received
0808 03:56:59.124323 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:59.124329 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:59.124484 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:59.124593 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:59.125476 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.126576 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.128174 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.128715 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.128809 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:59.134863 (+ 6054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:59.134877 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:59.134896 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:59.134932 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:59.138254 (+ 3322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.138920 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.138928 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.138931 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.139068 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.139391 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.139396 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.139399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.139858 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:59.140134 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:59.140442 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 03:56:59.140691 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":313,"threads_started":1}
I20260808 03:56:59.143628 27849 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } has no permanent_uuid. Determining permanent_uuid...
I20260808 03:56:59.144845 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.144384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:40515 (local address 127.0.0.1:50252)
0808 03:56:59.144599 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.144703 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 03:56:59.145151 27786 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
I20260808 03:56:59.146670 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.146329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:40515 (local address 127.0.0.1:50264)
0808 03:56:59.146479 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.146568 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 03:56:59.146888 27849 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:40515: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.188.keytab.
Entry for principal kudu/127.26.146.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.188.keytab.
I20260808 03:56:59.171380 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.188:40515
--webserver_interface=127.26.146.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.188.keytab
--principal=kudu/127.26.146.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:42361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:36417,127.26.146.189:33477,127.26.146.188:40515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:56:59.196211 27849 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } attempt: 1
I20260808 03:56:59.197522 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.196970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:40515 (local address 127.0.0.1:50266)
0808 03:56:59.197202 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.197324 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 03:56:59.198590 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.198178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:40515 (local address 127.0.0.1:50268)
0808 03:56:59.198341 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.198437 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 03:56:59.198792 27849 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:40515: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
I20260808 03:56:59.266764 27753 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } attempt: 5
I20260808 03:56:59.267027 27849 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } attempt: 2
I20260808 03:56:59.268232 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.267639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:40515 (local address 127.0.0.1:50284)
0808 03:56:59.267929 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.268029 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180}
I20260808 03:56:59.269470 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.269036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:40515 (local address 127.0.0.1:50296)
0808 03:56:59.269192 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.269296 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 03:56:59.269697 27849 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:40515: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
Aug 08 03:56:59 dist-test-slave-jlzn krb5kdc[27639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161418, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for kudu/127.26.146.189@KRBTEST.COM
I20260808 03:56:59.281565 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.267530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33477 (local address 127.0.0.1:45312)
0808 03:56:59.267899 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.267919 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:59.268052 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:59.268454 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:59.268460 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:59.269062 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:59.269314 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:59.269328 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.270876 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.270880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:59.271761 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:59.271774 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.272822 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.272827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:59.272999 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:59.273619 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:56:59.273639 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:59.275564 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:59.279269 (+ 3705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:59.279273 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:59.279276 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:59.279645 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:59.280071 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:59.280075 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:59.280077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:59.280223 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:59.280824 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:59.280827 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:59.280953 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 03:56:59.281065 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 03:56:59.281316 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220}
I20260808 03:56:59.281739 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.267689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45312 (local address 127.26.146.189:33477)
0808 03:56:59.267901 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:59.267907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:59.267980 (+ 73us) server_negotiation.cc:407] Connection header received
0808 03:56:59.268189 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:59.268193 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:59.268277 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:59.269194 (+ 917us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:59.269480 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.270717 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.271952 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.272639 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.272763 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:59.275759 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:59.275767 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:59.275771 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:59.275841 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:59.279143 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.279753 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.279759 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.279762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.279947 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.280320 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.280325 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.280328 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.280719 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:59.280862 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:59.281177 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 03:56:59.281403 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 03:56:59.282805 27753 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } has no permanent_uuid. Determining permanent_uuid...
I20260808 03:56:59.283780 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.283416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:40515 (local address 127.0.0.1:50310)
0808 03:56:59.283577 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.283657 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 03:56:59.284659 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.284342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:40515 (local address 127.0.0.1:50314)
0808 03:56:59.284488 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.284552 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 03:56:59.284840 27753 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:40515: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
I20260808 03:56:59.336079 27753 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } attempt: 1
I20260808 03:56:59.337399 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.336870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:40515 (local address 127.0.0.1:50322)
0808 03:56:59.337098 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.337224 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260808 03:56:59.338369 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.338032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:40515 (local address 127.0.0.1:50336)
0808 03:56:59.338184 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.338258 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 03:56:59.338585 27753 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:40515: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
I20260808 03:56:59.360917 27849 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } attempt: 3
I20260808 03:56:59.362109 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.361642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:40515 (local address 127.0.0.1:50352)
0808 03:56:59.361835 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.361940 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 03:56:59.363122 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.362699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:40515 (local address 127.0.0.1:50358)
0808 03:56:59.362919 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.363001 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
W20260808 03:56:59.363330 27849 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:40515: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
I20260808 03:56:59.406810 27753 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } attempt: 2
I20260808 03:56:59.408043 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.407487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:40515 (local address 127.0.0.1:50374)
0808 03:56:59.407731 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.407865 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 03:56:59.409107 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.408778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:40515 (local address 127.0.0.1:50390)
0808 03:56:59.408930 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.408999 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 03:56:59.409308 27753 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:40515: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
I20260808 03:56:59.506623 27849 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } attempt: 4
I20260808 03:56:59.507854 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.507343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:40515 (local address 127.0.0.1:50406)
0808 03:56:59.507564 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.507660 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 03:56:59.508783 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.508448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:40515 (local address 127.0.0.1:50416)
0808 03:56:59.508598 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.508675 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 03:56:59.508977 27849 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:40515: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
W20260808 03:56:59.513990 27863 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 03:56:59.514309 27863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:56:59.514407 27863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:56:59.514523 27863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 03:56:59.515488 27753 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } attempt: 3
I20260808 03:56:59.516414 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.516004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:40515 (local address 127.0.0.1:50420)
0808 03:56:59.516180 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.516256 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 03:56:59.517444 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.517068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:40515 (local address 127.0.0.1:50430)
0808 03:56:59.517264 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.517343 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260808 03:56:59.517653 27753 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:40515: Network error: Client connection negotiation failed: client connection to 127.26.146.188:40515: connect: Connection refused (error 111)
W20260808 03:56:59.522048 27863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:56:59.522153 27863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:56:59.522251 27863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:56:59.522317 27863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:56:59.522410 27863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:56:59.532306 27863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:42361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:36417,127.26.146.189:33477,127.26.146.188:40515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.188.keytab
--principal=kudu/127.26.146.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.188:40515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.146.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:56:59.533533 27863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:56:59.534843 27863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:56:59.546258 27876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:59.546316 27870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:56:59.546312 27871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:56:59.547299 27863 server_base.cc:1034] running on GCE node
I20260808 03:56:59.548123 27863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:56:59.549574 27863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:56:59.550798 27863 hybrid_clock.cc:648] HybridClock initialized: now 1786161419550727 us; error 39 us; skew 500 ppm
I20260808 03:56:59.551169 27863 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:56:59 dist-test-slave-jlzn krb5kdc[27639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161419, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:56:59.554370 27863 init.cc:377] Logged in from keytab as kudu/127.26.146.188@KRBTEST.COM (short username kudu)
I20260808 03:56:59.555795 27863 webserver.cc:460] Webserver started at http://127.26.146.188:39275/ using document root <none> and password file <none>
I20260808 03:56:59.556509 27863 fs_manager.cc:359] Metadata directory not provided
I20260808 03:56:59.556624 27863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:56:59.556929 27863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:56:59.559949 27863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-2/data/instance:
uuid: "f692f41cfff54fbaa0cce1e3eca94ee3"
format_stamp: "Formatted at 2026-08-08 03:56:59 on dist-test-slave-jlzn"
server_key: "ec61991e623aaa23f72b2c891ccc5a65"
server_key_iv: "e8c11f8056b03994443a9146334539d0"
server_key_version: "encryptionkey@0"
I20260808 03:56:59.560735 27863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-2/wal/instance:
uuid: "f692f41cfff54fbaa0cce1e3eca94ee3"
format_stamp: "Formatted at 2026-08-08 03:56:59 on dist-test-slave-jlzn"
server_key: "ec61991e623aaa23f72b2c891ccc5a65"
server_key_iv: "e8c11f8056b03994443a9146334539d0"
server_key_version: "encryptionkey@0"
I20260808 03:56:59.565914 27863 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:56:59.569332 27887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:56:59.570606 27863 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 03:56:59.570767 27863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-2/wal
uuid: "f692f41cfff54fbaa0cce1e3eca94ee3"
format_stamp: "Formatted at 2026-08-08 03:56:59 on dist-test-slave-jlzn"
server_key: "ec61991e623aaa23f72b2c891ccc5a65"
server_key_iv: "e8c11f8056b03994443a9146334539d0"
server_key_version: "encryptionkey@0"
I20260808 03:56:59.570964 27863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:56:59.587503 27863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:56:59.591532 27863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:56:59.591795 27863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:56:59.609177 27863 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.188:40515
I20260808 03:56:59.609198 27964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.188:40515 every 8 connection(s)
I20260808 03:56:59.610481 27863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-2/data/info.pb
I20260808 03:56:59.611913 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 27863
I20260808 03:56:59.612109 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786161058098901-27210-0/minicluster-data/master-2/wal/instance
I20260808 03:56:59.612684 27210 external_mini_cluster.cc:1428] Setting key c64bb33448108009dd0106a336e6704f
I20260808 03:56:59.615183 27965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:56:59.623837 27965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36417 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:56:59 dist-test-slave-jlzn krb5kdc[27639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.188@KRBTEST.COM
Aug 08 03:56:59 dist-test-slave-jlzn krb5kdc[27639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161419, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.188@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:56:59.636483 27969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.614528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50446 (local address 127.26.146.188:40515)
0808 03:56:59.615268 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:59.615282 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:59.615330 (+ 48us) server_negotiation.cc:407] Connection header received
0808 03:56:59.617562 (+ 2232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:59.617588 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:59.618043 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:59.618471 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:59.620015 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.621998 (+ 1983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.623253 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.624301 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.624482 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:59.628881 (+ 4399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:59.628892 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:59.628905 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:59.628944 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:59.632346 (+ 3402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.633097 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.633105 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.633122 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.633364 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.633728 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.633733 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.633736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.634419 (+ 683us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:59.634657 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:59.635041 (+ 384us) server_negotiation.cc:299] Negotiation successful
0808 03:56:59.635454 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":307,"threads_started":1}
I20260808 03:56:59.641721 27971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.626758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:36417 (local address 127.0.0.1:56354)
0808 03:56:59.627539 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.627581 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:59.627737 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:59.628154 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:59.628161 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:59.628768 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:59.629154 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:59.629167 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.630180 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.630186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:59.631835 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:59.631851 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.632536 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.632541 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:59.632677 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:59.633239 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:56:59.633267 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:59.635760 (+ 2493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:59.638995 (+ 3235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:59.639006 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:59.639012 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:59.639514 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:59.640042 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:59.640046 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:59.640050 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:59.640184 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:59.640697 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:59.640705 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:59.640874 (+ 169us) client_negotiation.cc:769] Sending connection context
0808 03:56:59.641034 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 03:56:59.641325 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":460,"threads_started":1}
I20260808 03:56:59.641788 27970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.626693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56354 (local address 127.26.146.190:36417)
0808 03:56:59.627273 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:59.627280 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:59.627680 (+ 400us) server_negotiation.cc:407] Connection header received
0808 03:56:59.627890 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:59.627896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:59.627973 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:59.628071 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:59.629306 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.630023 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.631976 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.632369 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.632487 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:59.635950 (+ 3463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:59.635956 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:59.635959 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:59.635991 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:59.638874 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.639664 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.639669 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.639673 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.639934 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.640288 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.640294 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.640297 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.640570 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:59.640728 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:59.641160 (+ 432us) server_negotiation.cc:299] Negotiation successful
0808 03:56:59.641354 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":272,"threads_started":1}
I20260808 03:56:59.643152 27965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:56:59 dist-test-slave-jlzn krb5kdc[27639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
Aug 08 03:56:59 dist-test-slave-jlzn krb5kdc[27639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161419, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.188@KRBTEST.COM for kudu/127.26.146.189@KRBTEST.COM
I20260808 03:56:59.653625 27972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.639368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56356 (local address 127.26.146.190:36417)
0808 03:56:59.640053 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:59.640057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:59.640077 (+ 20us) server_negotiation.cc:407] Connection header received
0808 03:56:59.640122 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:59.640126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:59.640176 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:59.640266 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:59.641477 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.642583 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.643834 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.644329 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.644437 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:59.648241 (+ 3804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:59.648246 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:59.648249 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:59.648278 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:59.651074 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.651729 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.651735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.651739 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.651913 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.652261 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.652266 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.652270 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.652529 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:59.652771 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:59.653041 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 03:56:59.653236 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":286,"threads_started":1}
I20260808 03:56:59.655467 27971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.643727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:33477 (local address 127.0.0.1:45314)
0808 03:56:59.643887 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.643903 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:59.643973 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:59.644400 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:59.644404 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:59.644737 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:59.644873 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:59.644881 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.645769 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.645775 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:59.646636 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:59.646647 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.647353 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.647359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:59.647521 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:59.648041 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:56:59.648056 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:59.650058 (+ 2002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:59.653032 (+ 2974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:59.653037 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:59.653041 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:59.653387 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:59.653827 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:59.653832 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:59.653835 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:59.654003 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:59.654595 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:59.654599 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:59.654751 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 03:56:59.654865 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 03:56:59.655136 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 03:56:59.655571 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.643987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45314 (local address 127.26.146.189:33477)
0808 03:56:59.644159 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:59.644163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:59.644177 (+ 14us) server_negotiation.cc:407] Connection header received
0808 03:56:59.644224 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:59.644228 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:59.644280 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:59.644358 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:59.645011 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.645644 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.646771 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.647176 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.647279 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:59.650234 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:59.650238 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:59.650242 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:59.650274 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:59.652910 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.653493 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.653499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.653502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.653685 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.654120 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.654125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.654128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.654459 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:59.654651 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:59.654984 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 03:56:59.655182 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 03:56:59.656718 27965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:56:59 dist-test-slave-jlzn krb5kdc[27639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161419, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.188@KRBTEST.COM for kudu/127.26.146.188@KRBTEST.COM
Aug 08 03:56:59 dist-test-slave-jlzn krb5kdc[27639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.189@KRBTEST.COM
I20260808 03:56:59.672189 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.655264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45326 (local address 127.26.146.189:33477)
0808 03:56:59.655898 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:59.655904 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:59.655920 (+ 16us) server_negotiation.cc:407] Connection header received
0808 03:56:59.655964 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:59.655969 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:59.656039 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:59.656129 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:59.657437 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.658382 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.660253 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.660921 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.661021 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:59.666070 (+ 5049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:59.666084 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:59.666089 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:59.666125 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:59.669550 (+ 3425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.670187 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.670193 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.670196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.670395 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.670801 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.670807 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.670810 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.671060 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:59.671289 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:59.671591 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 03:56:59.671788 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":15163,"threads_started":1}
I20260808 03:56:59.672210 27971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.657455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:40515 (local address 127.0.0.1:50448)
0808 03:56:59.657739 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.657757 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:59.657843 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:59.658220 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:59.658224 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:59.658637 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:59.658766 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:59.658774 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.659712 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.659717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:59.660858 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:59.660872 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.661862 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.661869 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:59.662097 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:59.662670 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:56:59.662693 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:59.665909 (+ 3216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:59.669651 (+ 3742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:59.669657 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:59.669661 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:59.670185 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:59.670560 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:59.670564 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:59.670567 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:59.670681 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:59.671205 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:59.671210 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:59.671369 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 03:56:59.671501 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 03:56:59.671789 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184}
I20260808 03:56:59.672381 27969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.657637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50448 (local address 127.26.146.188:40515)
0808 03:56:59.657796 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:59.657802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:59.657820 (+ 18us) server_negotiation.cc:407] Connection header received
0808 03:56:59.657998 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:59.658004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:59.658087 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:59.658185 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:59.658889 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.659600 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.661014 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.661695 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.661817 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:59.666124 (+ 4307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:59.666130 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:59.666135 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:59.666166 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:59.669509 (+ 3343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.670300 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.670305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.670308 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.670444 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.670792 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.670798 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.670802 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.671081 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:59.671217 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:59.671659 (+ 442us) server_negotiation.cc:299] Negotiation successful
0808 03:56:59.672002 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":57}
I20260808 03:56:59.677735 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:56:59.678162 27965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f692f41cfff54fbaa0cce1e3eca94ee3: Bootstrap starting.
I20260808 03:56:59.681808 27965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f692f41cfff54fbaa0cce1e3eca94ee3: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:59.683126 27965 log.cc:824] T 00000000000000000000000000000000 P f692f41cfff54fbaa0cce1e3eca94ee3: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:59.680606 27716 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "57ada18c2ae14dbeaf97008eee195b04"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56356
I20260808 03:56:59.686200 27965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f692f41cfff54fbaa0cce1e3eca94ee3: No bootstrap required, opened a new log
I20260808 03:56:59.693091 27965 raft_consensus.cc:348] T 00000000000000000000000000000000 P f692f41cfff54fbaa0cce1e3eca94ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ada18c2ae14dbeaf97008eee195b04" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36417 } } peers { permanent_uuid: "07f7751a8d4e41ea84d3405260a8fe1e" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } } peers { permanent_uuid: "f692f41cfff54fbaa0cce1e3eca94ee3" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } }
I20260808 03:56:59.693394 27965 raft_consensus.cc:729] T 00000000000000000000000000000000 P f692f41cfff54fbaa0cce1e3eca94ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f692f41cfff54fbaa0cce1e3eca94ee3, State: Initialized, Role: FOLLOWER
I20260808 03:56:59.693810 27965 consensus_queue.cc:260] T 00000000000000000000000000000000 P f692f41cfff54fbaa0cce1e3eca94ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ada18c2ae14dbeaf97008eee195b04" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36417 } } peers { permanent_uuid: "07f7751a8d4e41ea84d3405260a8fe1e" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } } peers { permanent_uuid: "f692f41cfff54fbaa0cce1e3eca94ee3" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } }
I20260808 03:56:59.694629 27976 sys_catalog.cc:455] T 00000000000000000000000000000000 P f692f41cfff54fbaa0cce1e3eca94ee3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ada18c2ae14dbeaf97008eee195b04" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36417 } } peers { permanent_uuid: "07f7751a8d4e41ea84d3405260a8fe1e" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } } peers { permanent_uuid: "f692f41cfff54fbaa0cce1e3eca94ee3" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } } }
I20260808 03:56:59.694861 27753 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } attempt: 4
I20260808 03:56:59.695019 27976 sys_catalog.cc:458] T 00000000000000000000000000000000 P f692f41cfff54fbaa0cce1e3eca94ee3 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:56:59.695956 27965 sys_catalog.cc:564] T 00000000000000000000000000000000 P f692f41cfff54fbaa0cce1e3eca94ee3 [sys.catalog]: configured and running, proceeding with master startup.
Aug 08 03:56:59 dist-test-slave-jlzn krb5kdc[27639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161418, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for kudu/127.26.146.188@KRBTEST.COM
W20260808 03:56:59.706234 27991 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f692f41cfff54fbaa0cce1e3eca94ee3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 03:56:59.706418 27991 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 03:56:59.712157 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.695439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:40515 (local address 127.0.0.1:50452)
0808 03:56:59.695718 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.695735 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:59.695851 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:59.696190 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:59.696195 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:59.696593 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:59.696761 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:59.696770 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.698112 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.698118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:59.699225 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:59.699239 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.700698 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.700702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:59.700913 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:59.701620 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:56:59.701644 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:59.704501 (+ 2857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:59.708912 (+ 4411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:59.708917 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:59.708921 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:59.709437 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:59.709879 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:59.709884 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:59.709888 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:59.710052 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:59.711102 (+ 1050us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:59.711107 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:59.711282 (+ 175us) client_negotiation.cc:769] Sending connection context
0808 03:56:59.711440 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 03:56:59.711726 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157}
I20260808 03:56:59.712677 27969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.695617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50452 (local address 127.26.146.188:40515)
0808 03:56:59.695839 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:59.695845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:59.695866 (+ 21us) server_negotiation.cc:407] Connection header received
0808 03:56:59.695987 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:59.695993 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:59.696055 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:59.696156 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:59.696899 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.697996 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.699399 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.700259 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.700396 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:59.704700 (+ 4304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:59.704706 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:59.704711 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:59.704747 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:59.708754 (+ 4007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.709580 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.709585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.709588 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.709758 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.710179 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.710184 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.710187 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.710955 (+ 768us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:59.711301 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:59.711648 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 03:56:59.712046 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 03:56:59.704857 27863 master_runner.cc:186] Error getting master registration for 127.26.146.190:36417: OK, instance_id { permanent_uuid: "57ada18c2ae14dbeaf97008eee195b04" instance_seqno: 1786161418554055 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 03:56:59.715874 27863 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 03:56:59.718461 27753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04: Bootstrap starting.
I20260808 03:56:59.722661 27753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:59.724018 27753 log.cc:824] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:59.727401 27753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04: No bootstrap required, opened a new log
I20260808 03:56:59.734999 27753 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ada18c2ae14dbeaf97008eee195b04" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36417 } } peers { permanent_uuid: "07f7751a8d4e41ea84d3405260a8fe1e" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } } peers { permanent_uuid: "f692f41cfff54fbaa0cce1e3eca94ee3" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } }
I20260808 03:56:59.735416 27753 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57ada18c2ae14dbeaf97008eee195b04, State: Initialized, Role: FOLLOWER
I20260808 03:56:59.735931 27753 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ada18c2ae14dbeaf97008eee195b04" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36417 } } peers { permanent_uuid: "07f7751a8d4e41ea84d3405260a8fe1e" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } } peers { permanent_uuid: "f692f41cfff54fbaa0cce1e3eca94ee3" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } }
I20260808 03:56:59.737079 27997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ada18c2ae14dbeaf97008eee195b04" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36417 } } peers { permanent_uuid: "07f7751a8d4e41ea84d3405260a8fe1e" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } } peers { permanent_uuid: "f692f41cfff54fbaa0cce1e3eca94ee3" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } } }
I20260808 03:56:59.737500 27997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:56:59.738605 27753 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:59.738677 27716 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 03:56:59.738886 27716 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:59.742079 27716 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ada18c2ae14dbeaf97008eee195b04" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36417 } } peers { permanent_uuid: "07f7751a8d4e41ea84d3405260a8fe1e" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } } peers { permanent_uuid: "f692f41cfff54fbaa0cce1e3eca94ee3" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } }
I20260808 03:56:59.744282 27716 leader_election.cc:288] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04 [CANDIDATE]: Term 1 election: Requested vote from peers 07f7751a8d4e41ea84d3405260a8fe1e (127.26.146.189:33477), f692f41cfff54fbaa0cce1e3eca94ee3 (127.26.146.188:40515)
I20260808 03:56:59.744228 27923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "57ada18c2ae14dbeaf97008eee195b04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f692f41cfff54fbaa0cce1e3eca94ee3"
I20260808 03:56:59.744735 27923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f692f41cfff54fbaa0cce1e3eca94ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:59.748126 27923 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f692f41cfff54fbaa0cce1e3eca94ee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57ada18c2ae14dbeaf97008eee195b04 in term 1.
I20260808 03:56:59.748826 27678 leader_election.cc:302] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57ada18c2ae14dbeaf97008eee195b04, f692f41cfff54fbaa0cce1e3eca94ee3; no voters:
I20260808 03:56:59.749321 27997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:56:59.762198 27997 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04 [term 1 LEADER]: Becoming Leader. State: Replica: 57ada18c2ae14dbeaf97008eee195b04, State: Running, Role: LEADER
I20260808 03:56:59.763473 27997 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ada18c2ae14dbeaf97008eee195b04" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36417 } } peers { permanent_uuid: "07f7751a8d4e41ea84d3405260a8fe1e" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } } peers { permanent_uuid: "f692f41cfff54fbaa0cce1e3eca94ee3" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } }
I20260808 03:56:59.780310 28006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57ada18c2ae14dbeaf97008eee195b04. Latest consensus state: current_term: 1 leader_uuid: "57ada18c2ae14dbeaf97008eee195b04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ada18c2ae14dbeaf97008eee195b04" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36417 } } peers { permanent_uuid: "07f7751a8d4e41ea84d3405260a8fe1e" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } } peers { permanent_uuid: "f692f41cfff54fbaa0cce1e3eca94ee3" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } } }
I20260808 03:56:59.781225 28006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:59.785650 28028 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260808 03:56:59.794764 27681 tablet.cc:2369] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:56:59.805528 27972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.771554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56370 (local address 127.26.146.190:36417)
0808 03:56:59.771864 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:59.771871 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:59.776667 (+ 4796us) server_negotiation.cc:407] Connection header received
0808 03:56:59.777681 (+ 1014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:59.777688 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:59.777770 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:59.777845 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:59.783110 (+ 5265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.784277 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.787912 (+ 3635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.788580 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.788677 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:59.792575 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:59.792580 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:59.792583 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:59.792615 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:59.798416 (+ 5801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.800240 (+ 1824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.800248 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.800251 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.800599 (+ 348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.801579 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.801586 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.801590 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.802358 (+ 768us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:59.802916 (+ 558us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:59.804275 (+ 1359us) server_negotiation.cc:299] Negotiation successful
0808 03:56:59.804751 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 03:56:59.808176 27849 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } attempt: 5
I20260808 03:56:59.815662 28028 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:56:59.818298 27969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.772156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50468 (local address 127.26.146.188:40515)
0808 03:56:59.772324 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:59.772329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:59.779835 (+ 7506us) server_negotiation.cc:407] Connection header received
0808 03:56:59.781359 (+ 1524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:59.781365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:59.781448 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:59.781538 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:59.788060 (+ 6522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.789271 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.791381 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.792116 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.792246 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:59.807322 (+ 15076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:59.807330 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:59.807334 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:59.807372 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:59.812232 (+ 4860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.813909 (+ 1677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.813918 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.813922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.814251 (+ 329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.814961 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.814966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.814998 (+ 32us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.815572 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:59.816064 (+ 492us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:59.817092 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0808 03:56:59.817557 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 03:56:59.822444 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.780612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45328 (local address 127.26.146.189:33477)
0808 03:56:59.780808 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:59.780814 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:59.793366 (+ 12552us) server_negotiation.cc:407] Connection header received
0808 03:56:59.793447 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:59.793453 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:59.793536 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:59.793641 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:59.798651 (+ 5010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.799718 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.803521 (+ 3803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.804174 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.804270 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:59.807311 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:59.807318 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:59.807322 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:59.807363 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:59.812943 (+ 5580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.814066 (+ 1123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.814072 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.814075 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.814239 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.814957 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.814962 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.814966 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.815348 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:59.815601 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:59.821837 (+ 6236us) server_negotiation.cc:299] Negotiation successful
0808 03:56:59.822086 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
Aug 08 03:56:59 dist-test-slave-jlzn krb5kdc[27639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161419, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.189@KRBTEST.COM for kudu/127.26.146.188@KRBTEST.COM
I20260808 03:56:59.841065 27923 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f692f41cfff54fbaa0cce1e3eca94ee3 [term 1 FOLLOWER]: Refusing update from remote peer 57ada18c2ae14dbeaf97008eee195b04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:56:59.842162 28006 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f692f41cfff54fbaa0cce1e3eca94ee3" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:56:59.842494 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.808771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:40515 (local address 127.0.0.1:50484)
0808 03:56:59.808962 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 03:56:59.808981 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 03:56:59.809092 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:56:59.819284 (+ 10192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:56:59.819291 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:56:59.819837 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:56:59.820084 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:59.820097 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.822850 (+ 2753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.822856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:59.824243 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:56:59.824257 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.826181 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.826188 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:56:59.826427 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:59.827261 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:56:59.827287 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:56:59.830232 (+ 2945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:56:59.839315 (+ 9083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:59.839321 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:59.839325 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:59.839897 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:59.840657 (+ 760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:56:59.840662 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:56:59.840665 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:56:59.840840 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:56:59.841520 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:56:59.841525 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:56:59.841678 (+ 153us) client_negotiation.cc:769] Sending connection context
0808 03:56:59.841818 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 03:56:59.842116 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 03:56:59.843653 27969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:56:59.809131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50484 (local address 127.26.146.188:40515)
0808 03:56:59.818747 (+ 9616us) server_negotiation.cc:206] Beginning negotiation
0808 03:56:59.818756 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:56:59.818775 (+ 19us) server_negotiation.cc:407] Connection header received
0808 03:56:59.818884 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:56:59.818890 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:56:59.819033 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:56:59.820623 (+ 1590us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:56:59.820937 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.822656 (+ 1719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.824505 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:56:59.825778 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:56:59.829205 (+ 3427us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:56:59.830751 (+ 1546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:56:59.830757 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:56:59.830762 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:56:59.830801 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:56:59.839133 (+ 8332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.840348 (+ 1215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.840353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.840356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.840509 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:56:59.840987 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:56:59.840991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:56:59.840994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:56:59.841377 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:56:59.842836 (+ 1459us) server_negotiation.cc:1035] Waiting for connection context
0808 03:56:59.842957 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 03:56:59.843204 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9422,"thread_start_us":19519,"threads_started":1}
W20260808 03:56:59.847388 27895 tablet.cc:2369] T 00000000000000000000000000000000 P f692f41cfff54fbaa0cce1e3eca94ee3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:56:59.849990 27849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07f7751a8d4e41ea84d3405260a8fe1e: Bootstrap starting.
I20260808 03:56:59.854998 27849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07f7751a8d4e41ea84d3405260a8fe1e: Neither blocks nor log segments found. Creating new log.
I20260808 03:56:59.856587 27849 log.cc:824] T 00000000000000000000000000000000 P 07f7751a8d4e41ea84d3405260a8fe1e: Log is configured to *not* fsync() on all Append() calls
I20260808 03:56:59.841338 27655 master_runner.cc:186] Error getting master registration for 127.26.146.189:33477: OK, instance_id { permanent_uuid: "07f7751a8d4e41ea84d3405260a8fe1e" instance_seqno: 1786161419081768 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 03:56:59.860359 27655 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 03:56:59.866138 27849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07f7751a8d4e41ea84d3405260a8fe1e: No bootstrap required, opened a new log
I20260808 03:56:59.873159 27976 sys_catalog.cc:455] T 00000000000000000000000000000000 P f692f41cfff54fbaa0cce1e3eca94ee3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57ada18c2ae14dbeaf97008eee195b04. Latest consensus state: current_term: 1 leader_uuid: "57ada18c2ae14dbeaf97008eee195b04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ada18c2ae14dbeaf97008eee195b04" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36417 } } peers { permanent_uuid: "07f7751a8d4e41ea84d3405260a8fe1e" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } } peers { permanent_uuid: "f692f41cfff54fbaa0cce1e3eca94ee3" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } } }
I20260808 03:56:59.873503 27976 sys_catalog.cc:458] T 00000000000000000000000000000000 P f692f41cfff54fbaa0cce1e3eca94ee3 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:56:59.875705 28037 mvcc.cc:205] Tried to move back new op lower bound from 7316117175640924160 to 7316117175362580480. Current Snapshot: MvccSnapshot[applied={T|T < 7316117175640924160}]
I20260808 03:56:59.876744 27849 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07f7751a8d4e41ea84d3405260a8fe1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ada18c2ae14dbeaf97008eee195b04" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36417 } } peers { permanent_uuid: "07f7751a8d4e41ea84d3405260a8fe1e" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } } peers { permanent_uuid: "f692f41cfff54fbaa0cce1e3eca94ee3" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } }
I20260808 03:56:59.877123 27849 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07f7751a8d4e41ea84d3405260a8fe1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f7751a8d4e41ea84d3405260a8fe1e, State: Initialized, Role: FOLLOWER
I20260808 03:56:59.877619 27849 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07f7751a8d4e41ea84d3405260a8fe1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ada18c2ae14dbeaf97008eee195b04" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36417 } } peers { permanent_uuid: "07f7751a8d4e41ea84d3405260a8fe1e" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } } peers { permanent_uuid: "f692f41cfff54fbaa0cce1e3eca94ee3" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } }
I20260808 03:56:59.879199 28038 mvcc.cc:205] Tried to move back new op lower bound from 7316117175640924160 to 7316117175362580480. Current Snapshot: MvccSnapshot[applied={T|T < 7316117175640924160}]
I20260808 03:56:59.891585 27997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "57ada18c2ae14dbeaf97008eee195b04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ada18c2ae14dbeaf97008eee195b04" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36417 } } peers { permanent_uuid: "07f7751a8d4e41ea84d3405260a8fe1e" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } } peers { permanent_uuid: "f692f41cfff54fbaa0cce1e3eca94ee3" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } } }
I20260808 03:56:59.891992 27997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:59.892167 27976 sys_catalog.cc:455] T 00000000000000000000000000000000 P f692f41cfff54fbaa0cce1e3eca94ee3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "57ada18c2ae14dbeaf97008eee195b04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ada18c2ae14dbeaf97008eee195b04" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36417 } } peers { permanent_uuid: "07f7751a8d4e41ea84d3405260a8fe1e" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } } peers { permanent_uuid: "f692f41cfff54fbaa0cce1e3eca94ee3" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } } }
I20260808 03:56:59.892424 27976 sys_catalog.cc:458] T 00000000000000000000000000000000 P f692f41cfff54fbaa0cce1e3eca94ee3 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:56:59.901968 28028 catalog_manager.cc:1347] Generated new cluster ID: 8bd18cf384c54aa0b118822f87431d88
I20260808 03:56:59.902103 28028 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:56:59.926923 27815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "57ada18c2ae14dbeaf97008eee195b04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "07f7751a8d4e41ea84d3405260a8fe1e"
I20260808 03:56:59.927381 27814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07f7751a8d4e41ea84d3405260a8fe1e [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:56:59.925786 28043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07f7751a8d4e41ea84d3405260a8fe1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ada18c2ae14dbeaf97008eee195b04" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36417 } } peers { permanent_uuid: "07f7751a8d4e41ea84d3405260a8fe1e" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } } peers { permanent_uuid: "f692f41cfff54fbaa0cce1e3eca94ee3" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } } }
I20260808 03:56:59.928237 27849 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07f7751a8d4e41ea84d3405260a8fe1e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:56:59.928676 28043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07f7751a8d4e41ea84d3405260a8fe1e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:56:59.932219 27814 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 07f7751a8d4e41ea84d3405260a8fe1e [term 1 FOLLOWER]: Refusing update from remote peer 57ada18c2ae14dbeaf97008eee195b04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:56:59.937723 28006 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f7751a8d4e41ea84d3405260a8fe1e" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:56:59.940107 27815 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 07f7751a8d4e41ea84d3405260a8fe1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57ada18c2ae14dbeaf97008eee195b04 in term 1.
I20260808 03:56:59.961148 28028 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:56:59.963032 28028 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:56:59.979861 28043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07f7751a8d4e41ea84d3405260a8fe1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57ada18c2ae14dbeaf97008eee195b04. Latest consensus state: current_term: 1 leader_uuid: "57ada18c2ae14dbeaf97008eee195b04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ada18c2ae14dbeaf97008eee195b04" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36417 } } peers { permanent_uuid: "07f7751a8d4e41ea84d3405260a8fe1e" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } } peers { permanent_uuid: "f692f41cfff54fbaa0cce1e3eca94ee3" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } } }
I20260808 03:56:59.980204 28043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07f7751a8d4e41ea84d3405260a8fe1e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:56:59.981256 28057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07f7751a8d4e41ea84d3405260a8fe1e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "57ada18c2ae14dbeaf97008eee195b04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ada18c2ae14dbeaf97008eee195b04" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36417 } } peers { permanent_uuid: "07f7751a8d4e41ea84d3405260a8fe1e" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } } peers { permanent_uuid: "f692f41cfff54fbaa0cce1e3eca94ee3" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } } }
I20260808 03:56:59.981741 28057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07f7751a8d4e41ea84d3405260a8fe1e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:56:59.981663 27997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "57ada18c2ae14dbeaf97008eee195b04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ada18c2ae14dbeaf97008eee195b04" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 36417 } } peers { permanent_uuid: "07f7751a8d4e41ea84d3405260a8fe1e" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 33477 } } peers { permanent_uuid: "f692f41cfff54fbaa0cce1e3eca94ee3" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 40515 } } }
I20260808 03:56:59.981971 27997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04 [sys.catalog]: This master's current role is: LEADER
I20260808 03:56:59.991422 28028 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57ada18c2ae14dbeaf97008eee195b04: Generated new TSK 0
I20260808 03:56:59.992702 28028 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:57:00.024159 28069 catalog_manager.cc:1259] Loaded cluster ID: 8bd18cf384c54aa0b118822f87431d88
I20260808 03:57:00.028656 28069 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 07f7751a8d4e41ea84d3405260a8fe1e: loading cluster ID for follower catalog manager: success
I20260808 03:57:00.032974 28069 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 07f7751a8d4e41ea84d3405260a8fe1e: acquiring CA information for follower catalog manager: success
I20260808 03:57:00.039522 28069 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 07f7751a8d4e41ea84d3405260a8fe1e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 03:57:00.120739 27210 mini_kdc.cc:348] Time spent kdestroy: real 0.110s user 0.000s sys 0.030s
W20260808 03:57:00.170328 27969 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:57:00.156419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50500 (local address 127.26.146.188:40515)
0808 03:57:00.156707 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:00.156714 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:00.165983 (+ 9269us) server_negotiation.cc:407] Connection header received
0808 03:57:00.168066 (+ 2083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:00.168073 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:00.168189 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:00.168285 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:00.170042 (+ 1757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50500: BlockingRecv error: recv got EOF from 127.0.0.1:50500 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 03:57:00.172170 27972 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:57:00.159974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56372 (local address 127.26.146.190:36417)
0808 03:57:00.160169 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:00.160176 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:00.166513 (+ 6337us) server_negotiation.cc:407] Connection header received
0808 03:57:00.168528 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:00.168534 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:00.168672 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:00.168751 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:00.171880 (+ 3129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56372: BlockingRecv error: recv got EOF from 127.0.0.1:56372 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260808 03:57:00.176926 27854 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:57:00.167073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45334 (local address 127.26.146.189:33477)
0808 03:57:00.167276 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:00.167282 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:00.173224 (+ 5942us) server_negotiation.cc:407] Connection header received
0808 03:57:00.173385 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:00.173391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:00.173484 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:00.173599 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:00.176669 (+ 3070us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45334: BlockingRecv error: recv got EOF from 127.0.0.1:45334 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 03:57:00.232218 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:00.216377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45338 (local address 127.26.146.189:33477)
0808 03:57:00.216591 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:00.216598 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:00.224196 (+ 7598us) server_negotiation.cc:407] Connection header received
0808 03:57:00.224794 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:00.224801 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:00.225036 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:00.225185 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:00.225927 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.227072 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.229086 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.229784 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.229887 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:00.230652 (+ 765us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:00.231367 (+ 715us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:00.231509 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:00.231756 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 03:57:00.231869 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260808 03:57:00.275507 27969 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:57:00.265475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50512 (local address 127.26.146.188:40515)
0808 03:57:00.265710 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:00.265717 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:00.273093 (+ 7376us) server_negotiation.cc:407] Connection header received
0808 03:57:00.273692 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:00.273701 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:00.273833 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:00.273911 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:00.275195 (+ 1284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50512: BlockingRecv error: recv got EOF from 127.0.0.1:50512 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 03:57:00.309047 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:00.302302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45354 (local address 127.26.146.189:33477)
0808 03:57:00.302515 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:00.302527 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:00.304361 (+ 1834us) server_negotiation.cc:407] Connection header received
0808 03:57:00.304580 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:00.304586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:00.304677 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:00.304765 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:00.305234 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.306030 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.307360 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.307866 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.307974 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:00.308352 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:00.308479 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:00.308552 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:00.308692 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 03:57:00.308779 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260808 03:57:00.321616 27969 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:57:00.315356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50522 (local address 127.26.146.188:40515)
0808 03:57:00.315512 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:00.315516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:00.320345 (+ 4829us) server_negotiation.cc:407] Connection header received
0808 03:57:00.320524 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:00.320527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:00.320607 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:00.320686 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:00.321433 (+ 747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50522: BlockingRecv error: recv got EOF from 127.0.0.1:50522 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 03:57:00.332388 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:00.326887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45358 (local address 127.26.146.189:33477)
0808 03:57:00.327043 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:00.327049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:00.327561 (+ 512us) server_negotiation.cc:407] Connection header received
0808 03:57:00.327750 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:00.327755 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:00.327859 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:00.327940 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:00.328347 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.329254 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.330653 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.331134 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.331280 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:00.331650 (+ 370us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:00.331764 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:00.331872 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:00.331998 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 03:57:00.332085 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 03:57:00.341596 27969 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:57:00.338325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50538 (local address 127.26.146.188:40515)
0808 03:57:00.338510 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:00.338516 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:00.340288 (+ 1772us) server_negotiation.cc:407] Connection header received
0808 03:57:00.340463 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:00.340468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:00.340539 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:00.340628 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:00.341408 (+ 780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50538: BlockingRecv error: recv got EOF from 127.0.0.1:50538 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 03:57:00.354660 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:00.348910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45366 (local address 127.26.146.189:33477)
0808 03:57:00.349412 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:00.349418 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:00.349906 (+ 488us) server_negotiation.cc:407] Connection header received
0808 03:57:00.350098 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:00.350104 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:00.350183 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:00.350264 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:00.350693 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.351676 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.353017 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.353458 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.353568 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:00.353969 (+ 401us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:00.354088 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:00.354161 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:00.354313 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 03:57:00.354389 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363}
W20260808 03:57:00.362293 27969 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:57:00.360298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50554 (local address 127.26.146.188:40515)
0808 03:57:00.360591 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:00.360597 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:00.361074 (+ 477us) server_negotiation.cc:407] Connection header received
0808 03:57:00.361216 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:00.361222 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:00.361298 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:00.361378 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:00.362083 (+ 705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50554: BlockingRecv error: recv got EOF from 127.0.0.1:50554 (error 108)
Metrics: {"server-negotiator.queue_time_us":159}
I20260808 03:57:00.378470 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:00.373092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45368 (local address 127.26.146.189:33477)
0808 03:57:00.373300 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:00.373307 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:00.373781 (+ 474us) server_negotiation.cc:407] Connection header received
0808 03:57:00.374034 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:00.374040 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:00.374110 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:00.374181 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:00.374593 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.375488 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.376806 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.377240 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.377351 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:00.377735 (+ 384us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:00.377860 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:00.377940 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:00.378077 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 03:57:00.378157 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260808 03:57:00.390375 27969 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:57:00.388237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50562 (local address 127.26.146.188:40515)
0808 03:57:00.388439 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:00.388447 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:00.388962 (+ 515us) server_negotiation.cc:407] Connection header received
0808 03:57:00.389157 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:00.389162 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:00.389244 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:00.389322 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:00.390111 (+ 789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50562: BlockingRecv error: recv got EOF from 127.0.0.1:50562 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 03:57:00.414835 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:00.409434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45374 (local address 127.26.146.189:33477)
0808 03:57:00.409647 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:00.409654 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:00.410159 (+ 505us) server_negotiation.cc:407] Connection header received
0808 03:57:00.410354 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:00.410358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:00.410429 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:00.410511 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:00.410944 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.411760 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.413120 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.413573 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.413683 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:00.414109 (+ 426us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:00.414222 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:00.414296 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:00.414450 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 03:57:00.414534 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260808 03:57:00.422650 27969 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:57:00.420497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50568 (local address 127.26.146.188:40515)
0808 03:57:00.420703 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:00.420709 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:00.421185 (+ 476us) server_negotiation.cc:407] Connection header received
0808 03:57:00.421354 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:00.421360 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:00.421436 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:00.421525 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:00.422429 (+ 904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50568: BlockingRecv error: recv got EOF from 127.0.0.1:50568 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 03:57:00.464497 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:00.458281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45386 (local address 127.26.146.189:33477)
0808 03:57:00.458514 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:00.458521 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:00.458938 (+ 417us) server_negotiation.cc:407] Connection header received
0808 03:57:00.459109 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:00.459115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:00.459231 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:00.459360 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:00.459773 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.460991 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.462328 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.463085 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.463229 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:00.463660 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:00.463833 (+ 173us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:00.463942 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:00.464054 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 03:57:00.464147 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260808 03:57:00.471930 27969 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:57:00.470108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50584 (local address 127.26.146.188:40515)
0808 03:57:00.470296 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:00.470302 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:00.470613 (+ 311us) server_negotiation.cc:407] Connection header received
0808 03:57:00.470771 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:00.470776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:00.470883 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:00.470987 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:00.471670 (+ 683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50584: BlockingRecv error: recv got EOF from 127.0.0.1:50584 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 03:57:00.556682 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:00.550817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45392 (local address 127.26.146.189:33477)
0808 03:57:00.550993 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:00.550999 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:00.551625 (+ 626us) server_negotiation.cc:407] Connection header received
0808 03:57:00.551823 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:00.551830 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:00.551923 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:00.552019 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:00.552551 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.553307 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.554790 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.555233 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.555332 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:00.555961 (+ 629us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:00.556071 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:00.556159 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:00.556303 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 03:57:00.556385 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 03:57:00.572193 27969 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:57:00.569849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50594 (local address 127.26.146.188:40515)
0808 03:57:00.570061 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:00.570069 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:00.570618 (+ 549us) server_negotiation.cc:407] Connection header received
0808 03:57:00.570825 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:00.570832 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:00.570935 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:00.571043 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:00.571974 (+ 931us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50594: BlockingRecv error: recv got EOF from 127.0.0.1:50594 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 03:57:00.708690 27991 catalog_manager.cc:1259] Loaded cluster ID: 8bd18cf384c54aa0b118822f87431d88
I20260808 03:57:00.708796 27991 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f692f41cfff54fbaa0cce1e3eca94ee3: loading cluster ID for follower catalog manager: success
I20260808 03:57:00.709697 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:00.704469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45396 (local address 127.26.146.189:33477)
0808 03:57:00.704732 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:00.704739 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:00.704992 (+ 253us) server_negotiation.cc:407] Connection header received
0808 03:57:00.705176 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:00.705181 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:00.705254 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:00.705333 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:00.705730 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.706394 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.707783 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.708460 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.708591 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:00.709005 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:00.709154 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:00.709250 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:00.709334 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 03:57:00.709421 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 03:57:00.712072 27991 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f692f41cfff54fbaa0cce1e3eca94ee3: acquiring CA information for follower catalog manager: success
I20260808 03:57:00.713899 27991 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f692f41cfff54fbaa0cce1e3eca94ee3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 03:57:00.721174 27969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:00.715728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50610 (local address 127.26.146.188:40515)
0808 03:57:00.715956 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:00.715962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:00.716238 (+ 276us) server_negotiation.cc:407] Connection header received
0808 03:57:00.716372 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:00.716376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:00.716496 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:00.716643 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:00.717001 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.717717 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.719055 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.719486 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.719622 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:00.720110 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:00.720654 (+ 544us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:00.720760 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:00.720851 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 03:57:00.720914 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 03:57:00.741171 27969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:00.727100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50616 (local address 127.26.146.188:40515)
0808 03:57:00.727291 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:00.727297 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:00.735061 (+ 7764us) server_negotiation.cc:407] Connection header received
0808 03:57:00.735461 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:00.735468 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:00.735574 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:00.735633 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:00.736344 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.737266 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.739016 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.739723 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.739883 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:00.740389 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:00.740517 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:00.740608 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:00.740780 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 03:57:00.740868 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 03:57:00.741693 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:00.728064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45412 (local address 127.26.146.189:33477)
0808 03:57:00.728286 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:00.728292 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:00.735701 (+ 7409us) server_negotiation.cc:407] Connection header received
0808 03:57:00.735981 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:00.735987 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:00.736084 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:00.736179 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:00.736874 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.737950 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.739779 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.740406 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.740509 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:00.741019 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:00.741116 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:00.741172 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:00.741372 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 03:57:00.741458 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 03:57:00.742913 28240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:00.727023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56388 (local address 127.26.146.190:36417)
0808 03:57:00.735871 (+ 8848us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:00.735877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:00.735896 (+ 19us) server_negotiation.cc:407] Connection header received
0808 03:57:00.736005 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:00.736010 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:00.736186 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:00.736383 (+ 197us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:00.736996 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.738104 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.740147 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:00.740679 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:00.741391 (+ 712us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:00.741608 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:00.742041 (+ 433us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:00.742159 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:00.742554 (+ 395us) server_negotiation.cc:299] Negotiation successful
0808 03:57:00.742640 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8692,"thread_start_us":8580,"threads_started":1}
I20260808 03:57:00.747771 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 27655
I20260808 03:57:00.748965 27746 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:57:00.859998 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 27655
I20260808 03:57:00.889873 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 27763
I20260808 03:57:00.890997 27840 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:57:00.998765 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 27763
I20260808 03:57:01.021361 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 27863
I20260808 03:57:01.022492 27958 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:57:01.128723 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 27863
2026-08-08T03:57:01Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4570 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:57:01 dist-test-slave-jlzn krb5kdc[28257](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:57:01 dist-test-slave-jlzn krb5kdc[28257](info): set up 2 sockets
Aug 08 03:57:01 dist-test-slave-jlzn krb5kdc[28257](info): commencing operation
krb5kdc: starting...
W20260808 03:57:02.346086 27210 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.002s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/krb5kdc/test-user.keytab.
Aug 08 03:57:02 dist-test-slave-jlzn krb5kdc[28257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:57:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:57:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
Entry for principal kudu/127.26.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab.
I20260808 03:57:02.667225 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:44159
--webserver_interface=127.26.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:34933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:44159,127.26.146.189:38067,127.26.146.188:36487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:57:03.015025 28272 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 03:57:03.015357 28272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:57:03.015461 28272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:57:03.015575 28272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:57:03.022754 28272 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:57:03.022837 28272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:57:03.022907 28272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:57:03.022977 28272 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:57:03.023059 28272 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:57:03.032714 28272 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:34933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:44159,127.26.146.189:38067,127.26.146.188:36487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.190.keytab
--principal=kudu/127.26.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.190:44159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:57:03.034039 28272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:57:03.035372 28272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:57:03.046886 28280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:57:03.046854 28283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:57:03.046815 28281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:57:03.047524 28272 server_base.cc:1034] running on GCE node
I20260808 03:57:03.048359 28272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:57:03.049734 28272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:57:03.051011 28272 hybrid_clock.cc:648] HybridClock initialized: now 1786161423050924 us; error 53 us; skew 500 ppm
I20260808 03:57:03.051342 28272 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:57:03 dist-test-slave-jlzn krb5kdc[28257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161423, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:57:03.054592 28272 init.cc:377] Logged in from keytab as kudu/127.26.146.190@KRBTEST.COM (short username kudu)
I20260808 03:57:03.056083 28272 webserver.cc:460] Webserver started at http://127.26.146.190:33369/ using document root <none> and password file <none>
I20260808 03:57:03.056753 28272 fs_manager.cc:359] Metadata directory not provided
I20260808 03:57:03.056845 28272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:57:03.057124 28272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:57:03.060081 28272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-0/data/instance:
uuid: "7627110361314a90abfa6c03debf9a5f"
format_stamp: "Formatted at 2026-08-08 03:57:03 on dist-test-slave-jlzn"
server_key: "8a268ee941f50edce590a32b60538d3d"
server_key_iv: "9cb418acd388874c27e40c92f037dcb3"
server_key_version: "encryptionkey@0"
I20260808 03:57:03.060842 28272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-0/wal/instance:
uuid: "7627110361314a90abfa6c03debf9a5f"
format_stamp: "Formatted at 2026-08-08 03:57:03 on dist-test-slave-jlzn"
server_key: "8a268ee941f50edce590a32b60538d3d"
server_key_iv: "9cb418acd388874c27e40c92f037dcb3"
server_key_version: "encryptionkey@0"
I20260808 03:57:03.065794 28272 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 03:57:03.069205 28291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:57:03.070492 28272 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:57:03.070662 28272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-0/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-0/wal
uuid: "7627110361314a90abfa6c03debf9a5f"
format_stamp: "Formatted at 2026-08-08 03:57:03 on dist-test-slave-jlzn"
server_key: "8a268ee941f50edce590a32b60538d3d"
server_key_iv: "9cb418acd388874c27e40c92f037dcb3"
server_key_version: "encryptionkey@0"
I20260808 03:57:03.070851 28272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:57:03.090827 28272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:57:03.094753 28272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:57:03.094975 28272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:57:03.112970 28272 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.190:44159
I20260808 03:57:03.112994 28359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.190:44159 every 8 connection(s)
I20260808 03:57:03.114333 28272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-0/data/info.pb
I20260808 03:57:03.118040 28361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:57:03.119647 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 28272
I20260808 03:57:03.119831 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-0/wal/instance
I20260808 03:57:03.120324 27210 external_mini_cluster.cc:1428] Setting key a00ca4c36bdf24f6cfba89014a79a717
I20260808 03:57:03.124511 28361 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44159 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:57:03 dist-test-slave-jlzn krb5kdc[28257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161423, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:57:03.146130 28366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.128938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47190 (local address 127.26.146.190:44159)
0808 03:57:03.129618 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:03.129629 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:03.129694 (+ 65us) server_negotiation.cc:407] Connection header received
0808 03:57:03.130888 (+ 1194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:03.130893 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:03.131140 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:03.131317 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:03.132616 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:03.133789 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:03.135121 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:03.135708 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:03.136044 (+ 336us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:03.139484 (+ 3440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:57:03.139492 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:57:03.139504 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:57:03.139540 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:57:03.142150 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:03.142924 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:03.142930 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:03.142932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:03.143104 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:03.143498 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:03.143501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:03.143504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:03.144056 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:57:03.144210 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:03.144813 (+ 603us) server_negotiation.cc:299] Negotiation successful
0808 03:57:03.145102 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":468,"thread_start_us":299,"threads_started":1}
I20260808 03:57:03.146158 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.127658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:44159 (local address 127.0.0.1:47190)
0808 03:57:03.128472 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.128529 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 03:57:03.130531 (+ 2002us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:57:03.131363 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:57:03.131370 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:57:03.132084 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:57:03.132417 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:57:03.132434 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:57:03.133936 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:57:03.133940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:57:03.134981 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:57:03.135003 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:57:03.136052 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:57:03.136060 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:57:03.136290 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:03.136946 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:57:03.136967 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:57:03.139255 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:57:03.142280 (+ 3025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:57:03.142291 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:57:03.142302 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:57:03.142801 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:57:03.143211 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:57:03.143215 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:57:03.143218 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:57:03.143406 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:57:03.144204 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:57:03.144211 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:57:03.144420 (+ 209us) client_negotiation.cc:769] Sending connection context
0808 03:57:03.144684 (+ 264us) client_negotiation.cc:240] Negotiation successful
0808 03:57:03.145284 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"mutex_wait_us":145,"thread_start_us":335,"threads_started":1}
I20260808 03:57:03.149325 28361 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } has no permanent_uuid. Determining permanent_uuid...
I20260808 03:57:03.150461 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.150085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:38067 (local address 127.0.0.1:42180)
0808 03:57:03.150251 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.150357 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:38067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 03:57:03.150741 28293 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.146.189:38067: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.26.146.189@KRBTEST.COM; defaulting to no policy
I20260808 03:57:03.152312 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.151938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:38067 (local address 127.0.0.1:42192)
0808 03:57:03.152081 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.152164 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:38067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 03:57:03.152499 28361 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.189:38067: Network error: Client connection negotiation failed: client connection to 127.26.146.189:38067: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.189.keytab.
Entry for principal kudu/127.26.146.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.189.keytab.
I20260808 03:57:03.190551 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.189:38067
--webserver_interface=127.26.146.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.189.keytab
--principal=kudu/127.26.146.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:34933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:44159,127.26.146.189:38067,127.26.146.188:36487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:57:03.201853 28361 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } attempt: 1
I20260808 03:57:03.203126 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.202588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:38067 (local address 127.0.0.1:42196)
0808 03:57:03.202810 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.202955 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:38067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 03:57:03.204267 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.203857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:38067 (local address 127.0.0.1:42202)
0808 03:57:03.204004 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.204092 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:38067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 03:57:03.204463 28361 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.189:38067: Network error: Client connection negotiation failed: client connection to 127.26.146.189:38067: connect: Connection refused (error 111)
I20260808 03:57:03.272735 28361 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } attempt: 2
I20260808 03:57:03.273873 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.273391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:38067 (local address 127.0.0.1:42210)
0808 03:57:03.273637 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.273729 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:38067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260808 03:57:03.274865 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.274515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:38067 (local address 127.0.0.1:42224)
0808 03:57:03.274664 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.274733 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:38067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 03:57:03.275046 28361 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.189:38067: Network error: Client connection negotiation failed: client connection to 127.26.146.189:38067: connect: Connection refused (error 111)
I20260808 03:57:03.366313 28361 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } attempt: 3
I20260808 03:57:03.367600 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.367088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:38067 (local address 127.0.0.1:42240)
0808 03:57:03.367323 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.367419 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:38067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260808 03:57:03.368729 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.368393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:38067 (local address 127.0.0.1:42252)
0808 03:57:03.368525 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.368595 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:38067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 03:57:03.368918 28361 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.189:38067: Network error: Client connection negotiation failed: client connection to 127.26.146.189:38067: connect: Connection refused (error 111)
I20260808 03:57:03.512157 28361 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } attempt: 4
I20260808 03:57:03.513444 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.512864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:38067 (local address 127.0.0.1:42266)
0808 03:57:03.513214 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.513305 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:38067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208}
I20260808 03:57:03.514672 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.514281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:38067 (local address 127.0.0.1:42282)
0808 03:57:03.514437 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.514526 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.189:38067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 03:57:03.514883 28361 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.189:38067: Network error: Client connection negotiation failed: client connection to 127.26.146.189:38067: connect: Connection refused (error 111)
W20260808 03:57:03.529107 28371 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 03:57:03.529489 28371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:57:03.529605 28371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:57:03.529717 28371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:57:03.537025 28371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:57:03.537132 28371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:57:03.537215 28371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:57:03.537281 28371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:57:03.537366 28371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:57:03.547238 28371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:34933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:44159,127.26.146.189:38067,127.26.146.188:36487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.189.keytab
--principal=kudu/127.26.146.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.189:38067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.146.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:57:03.548565 28371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:57:03.549863 28371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:57:03.561353 28380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:57:03.561302 28381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:57:03.561292 28383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:57:03.562570 28371 server_base.cc:1034] running on GCE node
I20260808 03:57:03.563340 28371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:57:03.564767 28371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:57:03.566006 28371 hybrid_clock.cc:648] HybridClock initialized: now 1786161423565925 us; error 58 us; skew 500 ppm
I20260808 03:57:03.566357 28371 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:57:03 dist-test-slave-jlzn krb5kdc[28257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161423, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:57:03.572947 28371 init.cc:377] Logged in from keytab as kudu/127.26.146.189@KRBTEST.COM (short username kudu)
I20260808 03:57:03.574523 28371 webserver.cc:460] Webserver started at http://127.26.146.189:42811/ using document root <none> and password file <none>
I20260808 03:57:03.575287 28371 fs_manager.cc:359] Metadata directory not provided
I20260808 03:57:03.575397 28371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:57:03.575670 28371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:57:03.578742 28371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-1/data/instance:
uuid: "31d63ab376764dacb7d53c495845655c"
format_stamp: "Formatted at 2026-08-08 03:57:03 on dist-test-slave-jlzn"
server_key: "b86db110275464a6973223d895d8a51d"
server_key_iv: "5e3b7d90a74530b1547c261f70d43a9c"
server_key_version: "encryptionkey@0"
I20260808 03:57:03.579528 28371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-1/wal/instance:
uuid: "31d63ab376764dacb7d53c495845655c"
format_stamp: "Formatted at 2026-08-08 03:57:03 on dist-test-slave-jlzn"
server_key: "b86db110275464a6973223d895d8a51d"
server_key_iv: "5e3b7d90a74530b1547c261f70d43a9c"
server_key_version: "encryptionkey@0"
I20260808 03:57:03.584705 28371 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260808 03:57:03.588088 28392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:57:03.589406 28371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 03:57:03.589558 28371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-1/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-1/wal
uuid: "31d63ab376764dacb7d53c495845655c"
format_stamp: "Formatted at 2026-08-08 03:57:03 on dist-test-slave-jlzn"
server_key: "b86db110275464a6973223d895d8a51d"
server_key_iv: "5e3b7d90a74530b1547c261f70d43a9c"
server_key_version: "encryptionkey@0"
I20260808 03:57:03.589740 28371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:57:03.627115 28371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:57:03.631520 28371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:57:03.631757 28371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:57:03.649418 28371 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.189:38067
I20260808 03:57:03.649437 28470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.189:38067 every 8 connection(s)
I20260808 03:57:03.650782 28371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-1/data/info.pb
I20260808 03:57:03.652900 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 28371
I20260808 03:57:03.653074 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-1/wal/instance
I20260808 03:57:03.653504 27210 external_mini_cluster.cc:1428] Setting key 92479b3a0d7e4e8cbd1809f2bff28f37
I20260808 03:57:03.654556 28471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:57:03.662367 28471 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44159 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:57:03 dist-test-slave-jlzn krb5kdc[28257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161423, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.189@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.146.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.146.188@KRBTEST.COM" created.
I20260808 03:57:03.690220 28474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.665726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47196 (local address 127.26.146.190:44159)
0808 03:57:03.668197 (+ 2471us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:03.668204 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:03.668961 (+ 757us) server_negotiation.cc:407] Connection header received
0808 03:57:03.671412 (+ 2451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:03.671418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:03.671533 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:03.671681 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:03.673825 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:03.674944 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:03.677408 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:03.677944 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:03.678073 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:03.682714 (+ 4641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:57:03.682721 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:57:03.682727 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:57:03.682786 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:57:03.686458 (+ 3672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:03.687324 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:03.687331 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:03.687335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:03.687499 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:03.688055 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:03.688061 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:03.688064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:03.688497 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:57:03.688667 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:03.689568 (+ 901us) server_negotiation.cc:299] Negotiation successful
0808 03:57:03.689817 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2327,"thread_start_us":2203,"threads_started":1}
I20260808 03:57:03.691418 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.666329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:44159 (local address 127.0.0.1:47196)
0808 03:57:03.668575 (+ 2246us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.668637 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 03:57:03.671078 (+ 2441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:57:03.671970 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:57:03.671983 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:57:03.672763 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:57:03.673550 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:57:03.673577 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:57:03.675149 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:57:03.675155 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:57:03.677245 (+ 2090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:57:03.677261 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:57:03.678190 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:57:03.678199 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:57:03.678493 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:03.679273 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:57:03.679306 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:57:03.682381 (+ 3075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:57:03.686634 (+ 4253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:57:03.686643 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:57:03.686656 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:57:03.687177 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:57:03.687709 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:57:03.687713 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:57:03.687715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:57:03.687941 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:57:03.688647 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:57:03.688655 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:57:03.689136 (+ 481us) client_negotiation.cc:769] Sending connection context
0808 03:57:03.689438 (+ 302us) client_negotiation.cc:240] Negotiation successful
0808 03:57:03.689968 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1951,"thread_start_us":1754,"threads_started":1}
I20260808 03:57:03.693564 28471 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:57:03 dist-test-slave-jlzn krb5kdc[28257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161423, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.189@KRBTEST.COM for kudu/127.26.146.189@KRBTEST.COM
I20260808 03:57:03.719396 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.694600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:38067 (local address 127.0.0.1:42298)
0808 03:57:03.694806 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.694826 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:57:03.694936 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:57:03.704949 (+ 10013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:57:03.704954 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:57:03.705630 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:57:03.705842 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:57:03.705855 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:57:03.708152 (+ 2297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:57:03.708158 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:57:03.709346 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:57:03.709361 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:57:03.710469 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:57:03.710475 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:57:03.710634 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:03.711317 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:57:03.711337 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:57:03.713540 (+ 2203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:57:03.716937 (+ 3397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:57:03.716943 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:57:03.716948 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:57:03.717286 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:57:03.717654 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:57:03.717659 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:57:03.717663 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:57:03.717800 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:57:03.718426 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:57:03.718431 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:57:03.718587 (+ 156us) client_negotiation.cc:769] Sending connection context
0808 03:57:03.718708 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 03:57:03.718986 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 03:57:03.719480 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.694980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42298 (local address 127.26.146.189:38067)
0808 03:57:03.704352 (+ 9372us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:03.704361 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:03.704397 (+ 36us) server_negotiation.cc:407] Connection header received
0808 03:57:03.704473 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:03.704482 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:03.704768 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:03.706048 (+ 1280us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:03.706306 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:03.708008 (+ 1702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:03.709520 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:03.710325 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:03.710943 (+ 618us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:03.713737 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:57:03.713749 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:57:03.713764 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:57:03.713792 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:57:03.716804 (+ 3012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:03.717410 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:03.717417 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:03.717419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:03.717536 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:03.717900 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:03.717903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:03.717905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:03.718280 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:57:03.718481 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:03.718883 (+ 402us) server_negotiation.cc:299] Negotiation successful
0808 03:57:03.719119 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9164,"thread_start_us":9023,"threads_started":1}
I20260808 03:57:03.722239 28471 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } has no permanent_uuid. Determining permanent_uuid...
I20260808 03:57:03.723340 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.722832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:36487 (local address 127.0.0.1:53172)
0808 03:57:03.723055 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.723172 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 03:57:03.723645 28398 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
I20260808 03:57:03.725262 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.724911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:36487 (local address 127.0.0.1:53184)
0808 03:57:03.725064 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.725152 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 03:57:03.725557 28471 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:36487: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.146.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.188.keytab.
Entry for principal kudu/127.26.146.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.188.keytab.
I20260808 03:57:03.736840 27210 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
/tmp/dist-test-taskT_CS6X/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.188:36487
--webserver_interface=127.26.146.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.188.keytab
--principal=kudu/127.26.146.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.146.148:34933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.146.190:44159,127.26.146.189:38067,127.26.146.188:36487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:57:03.774858 28471 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } attempt: 1
I20260808 03:57:03.776139 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.775614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:36487 (local address 127.0.0.1:53192)
0808 03:57:03.775845 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.775963 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 03:57:03.777215 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.776844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:36487 (local address 127.0.0.1:53196)
0808 03:57:03.777020 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.777103 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 03:57:03.777498 28471 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:36487: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
I20260808 03:57:03.814131 28361 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } attempt: 5
Aug 08 03:57:03 dist-test-slave-jlzn krb5kdc[28257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161423, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for kudu/127.26.146.189@KRBTEST.COM
I20260808 03:57:03.827107 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.814852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:38067 (local address 127.0.0.1:42304)
0808 03:57:03.815046 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.815068 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 03:57:03.815237 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:57:03.815610 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:57:03.815615 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:57:03.816073 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:57:03.816271 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:57:03.816281 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:57:03.817285 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:57:03.817291 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:57:03.818152 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:57:03.818165 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:57:03.818937 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:57:03.818943 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:57:03.819147 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:03.819745 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:57:03.819763 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:57:03.821843 (+ 2080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:57:03.824798 (+ 2955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:57:03.824804 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:57:03.824809 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:57:03.825179 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:57:03.825525 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:57:03.825531 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:57:03.825534 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:57:03.825668 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:57:03.826177 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:57:03.826182 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:57:03.826321 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 03:57:03.826443 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 03:57:03.826753 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 03:57:03.827107 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.815083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42304 (local address 127.26.146.189:38067)
0808 03:57:03.815286 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:03.815292 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:03.815314 (+ 22us) server_negotiation.cc:407] Connection header received
0808 03:57:03.815378 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:03.815383 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:03.815465 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:03.815548 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:03.816428 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:03.817146 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:03.818300 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:03.818774 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:03.818890 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:03.822034 (+ 3144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:57:03.822039 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:57:03.822043 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:57:03.822076 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:57:03.824665 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:03.825287 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:03.825293 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:03.825297 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:03.825421 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:03.825769 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:03.825774 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:03.825778 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:03.826058 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:57:03.826190 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:03.826556 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 03:57:03.826776 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 03:57:03.828564 28361 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } has no permanent_uuid. Determining permanent_uuid...
I20260808 03:57:03.829540 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.829182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:36487 (local address 127.0.0.1:53212)
0808 03:57:03.829338 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.829410 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 03:57:03.830348 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.830046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:36487 (local address 127.0.0.1:53214)
0808 03:57:03.830187 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.830255 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 03:57:03.830530 28361 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:36487: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
I20260808 03:57:03.845705 28471 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } attempt: 2
I20260808 03:57:03.846699 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.846272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:36487 (local address 127.0.0.1:53216)
0808 03:57:03.846454 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.846539 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 03:57:03.847733 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.847401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:36487 (local address 127.0.0.1:53230)
0808 03:57:03.847527 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.847602 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 03:57:03.847940 28471 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:36487: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
I20260808 03:57:03.881770 28361 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } attempt: 1
I20260808 03:57:03.882910 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.882418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:36487 (local address 127.0.0.1:53244)
0808 03:57:03.882635 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.882729 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 03:57:03.884058 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.883675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:36487 (local address 127.0.0.1:53254)
0808 03:57:03.883875 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.883945 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 03:57:03.884276 28361 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:36487: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
I20260808 03:57:03.939193 28471 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } attempt: 3
I20260808 03:57:03.940501 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.940006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:36487 (local address 127.0.0.1:53270)
0808 03:57:03.940227 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.940326 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 03:57:03.941591 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.941158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:36487 (local address 127.0.0.1:53280)
0808 03:57:03.941379 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.941470 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
W20260808 03:57:03.941792 28471 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:36487: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
I20260808 03:57:03.952461 28361 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } attempt: 2
I20260808 03:57:03.953459 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.953007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:36487 (local address 127.0.0.1:53282)
0808 03:57:03.953225 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.953307 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 03:57:03.954399 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:03.954091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:36487 (local address 127.0.0.1:53286)
0808 03:57:03.954239 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:03.954303 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 03:57:03.954591 28361 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:36487: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
I20260808 03:57:04.060786 28361 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } attempt: 3
I20260808 03:57:04.062067 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:04.061517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:36487 (local address 127.0.0.1:53300)
0808 03:57:04.061768 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:04.061861 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260808 03:57:04.063205 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:04.062811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:36487 (local address 127.0.0.1:53314)
0808 03:57:04.063006 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:04.063082 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260808 03:57:04.063414 28361 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:36487: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
I20260808 03:57:04.084996 28471 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } attempt: 4
I20260808 03:57:04.086092 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:04.085616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:36487 (local address 127.0.0.1:53318)
0808 03:57:04.085833 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:04.085919 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 03:57:04.086980 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:04.086668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:36487 (local address 127.0.0.1:53334)
0808 03:57:04.086809 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:04.086878 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 03:57:04.087181 28471 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.146.188:36487: Network error: Client connection negotiation failed: client connection to 127.26.146.188:36487: connect: Connection refused (error 111)
W20260808 03:57:04.097572 28483 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 03:57:04.097908 28483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:57:04.098009 28483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:57:04.098121 28483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:57:04.105329 28483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:57:04.105425 28483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:57:04.105526 28483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:57:04.105593 28483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:57:04.105680 28483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:57:04.117709 28483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.146.148:34933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.146.190:44159,127.26.146.189:38067,127.26.146.188:36487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/krb5kdc/kudu_127.26.146.188.keytab
--principal=kudu/127.26.146.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.146.188:36487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.146.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 02:38:26 UTC on ccycloud.quasar-koaxys.root.comops.site
build id 4574019
ASAN enabled
I20260808 03:57:04.119297 28483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:57:04.121042 28483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:57:04.132799 28493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:57:04.132840 28488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:57:04.132870 28489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:57:04.133998 28483 server_base.cc:1034] running on GCE node
I20260808 03:57:04.134801 28483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:57:04.136201 28483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:57:04.137432 28483 hybrid_clock.cc:648] HybridClock initialized: now 1786161424137358 us; error 44 us; skew 500 ppm
I20260808 03:57:04.137805 28483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:57:04 dist-test-slave-jlzn krb5kdc[28257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161424, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:57:04.140928 28483 init.cc:377] Logged in from keytab as kudu/127.26.146.188@KRBTEST.COM (short username kudu)
I20260808 03:57:04.142349 28483 webserver.cc:460] Webserver started at http://127.26.146.188:36737/ using document root <none> and password file <none>
I20260808 03:57:04.143015 28483 fs_manager.cc:359] Metadata directory not provided
I20260808 03:57:04.143117 28483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:57:04.143419 28483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:57:04.146346 28483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-2/data/instance:
uuid: "837832b4e7124c42a52aed11f7ae7b63"
format_stamp: "Formatted at 2026-08-08 03:57:04 on dist-test-slave-jlzn"
server_key: "0a761829efb2c6bd4c1caa70d12c8941"
server_key_iv: "96d83a855387302bc9cbef846aa84ca3"
server_key_version: "encryptionkey@0"
I20260808 03:57:04.147127 28483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-2/wal/instance:
uuid: "837832b4e7124c42a52aed11f7ae7b63"
format_stamp: "Formatted at 2026-08-08 03:57:04 on dist-test-slave-jlzn"
server_key: "0a761829efb2c6bd4c1caa70d12c8941"
server_key_iv: "96d83a855387302bc9cbef846aa84ca3"
server_key_version: "encryptionkey@0"
I20260808 03:57:04.152384 28483 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 03:57:04.155858 28503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:57:04.157142 28483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:57:04.157315 28483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-2/data,/tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-2/wal
uuid: "837832b4e7124c42a52aed11f7ae7b63"
format_stamp: "Formatted at 2026-08-08 03:57:04 on dist-test-slave-jlzn"
server_key: "0a761829efb2c6bd4c1caa70d12c8941"
server_key_iv: "96d83a855387302bc9cbef846aa84ca3"
server_key_version: "encryptionkey@0"
I20260808 03:57:04.157558 28483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:57:04.175422 28483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:57:04.179937 28483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:57:04.180192 28483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:57:04.197937 28483 rpc_server.cc:307] RPC server started. Bound to: 127.26.146.188:36487
I20260808 03:57:04.197958 28583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.146.188:36487 every 8 connection(s)
I20260808 03:57:04.199283 28483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-2/data/info.pb
I20260808 03:57:04.200865 27210 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu as pid 28483
I20260808 03:57:04.201051 27210 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT_CS6X/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786161058098901-27210-0/minicluster-data/master-2/wal/instance
I20260808 03:57:04.201674 27210 external_mini_cluster.cc:1428] Setting key 205c3203c598ec976636805afb06a36b
I20260808 03:57:04.203923 28584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:57:04.211033 28584 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44159 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:57:04 dist-test-slave-jlzn krb5kdc[28257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.188@KRBTEST.COM
Aug 08 03:57:04 dist-test-slave-jlzn krb5kdc[28257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161424, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.188@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
I20260808 03:57:04.223718 28591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:04.203467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53348 (local address 127.26.146.188:36487)
0808 03:57:04.204274 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:04.204286 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:04.204326 (+ 40us) server_negotiation.cc:407] Connection header received
0808 03:57:04.205904 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:04.205925 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:04.206296 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:04.206646 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:04.208221 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.209576 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.210800 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.211483 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.211669 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:04.216452 (+ 4783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:57:04.216462 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:57:04.216475 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:57:04.216518 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:57:04.219892 (+ 3374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:04.220526 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:04.220533 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:04.220538 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:04.220722 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:04.221089 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:04.221093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:04.221095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:04.221555 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:57:04.221867 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:04.222343 (+ 476us) server_negotiation.cc:299] Negotiation successful
0808 03:57:04.222695 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":351,"threads_started":1}
I20260808 03:57:04.227978 28593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:04.213613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.190:44159 (local address 127.0.0.1:47212)
0808 03:57:04.214343 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:04.214415 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0808 03:57:04.214568 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:57:04.214972 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:57:04.214980 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:57:04.215719 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:57:04.216077 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:57:04.216090 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.217411 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.217415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:57:04.218632 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:57:04.218647 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.219572 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.219576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:57:04.219699 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:04.220242 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:57:04.220258 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:57:04.222187 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:57:04.225368 (+ 3181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:57:04.225379 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:57:04.225386 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:57:04.225798 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:57:04.226170 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:57:04.226173 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:57:04.226175 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:57:04.226313 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:57:04.226940 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:57:04.226946 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:57:04.227101 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 03:57:04.227249 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 03:57:04.227557 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":414,"threads_started":1}
I20260808 03:57:04.228086 28592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:04.213559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47212 (local address 127.26.146.190:44159)
0808 03:57:04.214034 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:04.214041 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:04.214508 (+ 467us) server_negotiation.cc:407] Connection header received
0808 03:57:04.214716 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:04.214721 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:04.214799 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:04.214890 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:04.216240 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.217251 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.218802 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.219397 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.219543 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:04.222366 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:57:04.222372 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:57:04.222375 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:57:04.222415 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:57:04.225235 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:04.225911 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:04.225915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:04.225917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:04.226051 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:04.226435 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:04.226439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:04.226442 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:04.226808 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:57:04.226970 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:04.227416 (+ 446us) server_negotiation.cc:299] Negotiation successful
0808 03:57:04.227689 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":248,"threads_started":1}
I20260808 03:57:04.229389 28584 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:57:04 dist-test-slave-jlzn krb5kdc[28257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.190@KRBTEST.COM
Aug 08 03:57:04 dist-test-slave-jlzn krb5kdc[28257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161424, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.188@KRBTEST.COM for kudu/127.26.146.189@KRBTEST.COM
I20260808 03:57:04.240556 28594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:04.226446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47222 (local address 127.26.146.190:44159)
0808 03:57:04.227037 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:04.227042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:04.227062 (+ 20us) server_negotiation.cc:407] Connection header received
0808 03:57:04.227098 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:04.227101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:04.227150 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:04.227236 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:04.228560 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.229600 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.230985 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.231401 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.231524 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:04.235363 (+ 3839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:57:04.235368 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:57:04.235371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:57:04.235402 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:57:04.238003 (+ 2601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:04.238607 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:04.238613 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:04.238617 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:04.238736 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:04.239069 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:04.239081 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:04.239085 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:04.239351 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:57:04.239598 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:04.239950 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 03:57:04.240169 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":271,"threads_started":1}
I20260808 03:57:04.240610 28361 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } attempt: 4
I20260808 03:57:04.241956 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:04.230194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42312 (local address 127.26.146.189:38067)
0808 03:57:04.230363 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:04.230369 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:04.230388 (+ 19us) server_negotiation.cc:407] Connection header received
0808 03:57:04.230560 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:04.230565 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:04.230629 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:04.230710 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:04.231378 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.232181 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.233315 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.233733 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.233840 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:04.236762 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:57:04.236766 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:57:04.236769 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:57:04.236807 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:57:04.239286 (+ 2479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:04.239892 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:04.239898 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:04.239901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:04.240040 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:04.240430 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:04.240435 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:04.240438 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:04.240788 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:57:04.240952 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:04.241386 (+ 434us) server_negotiation.cc:299] Negotiation successful
0808 03:57:04.241605 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 03:57:04.242509 28593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:04.230169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.189:38067 (local address 127.0.0.1:42312)
0808 03:57:04.230329 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:04.230347 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 03:57:04.230437 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:57:04.230742 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:57:04.230747 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:57:04.231100 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:57:04.231233 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:57:04.231241 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.232308 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.232314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:57:04.233178 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:57:04.233189 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.233871 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.233877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:57:04.234054 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:04.234536 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:57:04.234557 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:57:04.236579 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:57:04.239409 (+ 2830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:57:04.239414 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:57:04.239418 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:57:04.239756 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:57:04.240145 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:57:04.240150 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:57:04.240154 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:57:04.240304 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:57:04.240924 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:57:04.240929 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:57:04.241104 (+ 175us) client_negotiation.cc:769] Sending connection context
0808 03:57:04.241912 (+ 808us) client_negotiation.cc:240] Negotiation successful
0808 03:57:04.242197 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 03:57:04.244035 28584 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:57:04 dist-test-slave-jlzn krb5kdc[28257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161423, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.190@KRBTEST.COM for kudu/127.26.146.188@KRBTEST.COM
Aug 08 03:57:04 dist-test-slave-jlzn krb5kdc[28257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.146.189@KRBTEST.COM
I20260808 03:57:04.254400 28591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:04.241332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53358 (local address 127.26.146.188:36487)
0808 03:57:04.241920 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:04.241925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:04.241942 (+ 17us) server_negotiation.cc:407] Connection header received
0808 03:57:04.241986 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:04.241991 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:04.242060 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:04.242144 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:04.243085 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.243960 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.245432 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.245917 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.246019 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:04.249344 (+ 3325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:57:04.249349 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:57:04.249352 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:57:04.249380 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:57:04.251966 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:04.252551 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:04.252557 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:04.252561 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:04.252693 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:04.253047 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:04.253052 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:04.253055 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:04.253364 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:57:04.253488 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:04.253865 (+ 377us) server_negotiation.cc:299] Negotiation successful
0808 03:57:04.254072 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477}
I20260808 03:57:04.254410 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:04.241246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:36487 (local address 127.0.0.1:53358)
0808 03:57:04.241594 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:04.241618 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 03:57:04.241722 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:57:04.242315 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:57:04.242319 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:57:04.242764 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:57:04.242958 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:57:04.242970 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.244085 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.244091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:57:04.245314 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:57:04.245326 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.246051 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.246057 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:57:04.246247 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:04.246853 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:57:04.246877 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:57:04.249165 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:57:04.252087 (+ 2922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:57:04.252093 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:57:04.252097 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:57:04.252448 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:57:04.252808 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:57:04.252814 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:57:04.252818 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:57:04.252949 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:57:04.253475 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:57:04.253480 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:57:04.253639 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 03:57:04.253756 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 03:57:04.254029 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157}
I20260808 03:57:04.261212 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:04.242781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42316 (local address 127.26.146.189:38067)
0808 03:57:04.242948 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:04.242953 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:04.242970 (+ 17us) server_negotiation.cc:407] Connection header received
0808 03:57:04.243025 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:04.243030 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:04.243102 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:04.243186 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:04.245075 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.246201 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.247914 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.248616 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.248738 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:04.254390 (+ 5652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:57:04.254397 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:57:04.254403 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:57:04.254445 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:57:04.257989 (+ 3544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:04.258868 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:04.258874 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:04.258878 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:04.259062 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:04.259521 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:04.259527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:04.259531 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:04.259921 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:57:04.260186 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:04.260654 (+ 468us) server_negotiation.cc:299] Negotiation successful
0808 03:57:04.260866 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 03:57:04.261844 28361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f: Bootstrap starting.
Aug 08 03:57:04 dist-test-slave-jlzn krb5kdc[28257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161424, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.188@KRBTEST.COM for kudu/127.26.146.188@KRBTEST.COM
I20260808 03:57:04.266206 28361 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f: Neither blocks nor log segments found. Creating new log.
I20260808 03:57:04.266484 27210 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:57:04.267409 28361 log.cc:824] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f: Log is configured to *not* fsync() on all Append() calls
I20260808 03:57:04.269522 28593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:04.244773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:36487 (local address 127.0.0.1:53374)
0808 03:57:04.245008 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:04.245024 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:57:04.245116 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:57:04.255252 (+ 10136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:57:04.255258 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:57:04.255734 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:57:04.255943 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:57:04.255954 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.257296 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.257301 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:57:04.258587 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:57:04.258601 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.259680 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.259686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:57:04.259955 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:04.260478 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:57:04.260499 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:57:04.263351 (+ 2852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:57:04.266859 (+ 3508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:57:04.266864 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:57:04.266867 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:57:04.267337 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:57:04.267735 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:57:04.267740 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:57:04.267744 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:57:04.267921 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:57:04.268543 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:57:04.268547 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:57:04.268698 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 03:57:04.268847 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 03:57:04.269150 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260808 03:57:04.269625 28591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:04.244898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53374 (local address 127.26.146.188:36487)
0808 03:57:04.254873 (+ 9975us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:04.254879 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:04.254899 (+ 20us) server_negotiation.cc:407] Connection header received
0808 03:57:04.254942 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:04.254947 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:04.255017 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:04.255120 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:04.256122 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.257147 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.258719 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.259358 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.259444 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:04.263542 (+ 4098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:57:04.263549 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:57:04.263554 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:57:04.263589 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:57:04.266733 (+ 3144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:04.267467 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:04.267472 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:04.267475 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:04.267626 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:04.268029 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:04.268033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:04.268037 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:04.268350 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:57:04.268455 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:04.268971 (+ 516us) server_negotiation.cc:299] Negotiation successful
0808 03:57:04.269290 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":118,"server-negotiator.queue_time_us":9853,"thread_start_us":15201,"threads_started":1}
I20260808 03:57:04.271023 28361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f: No bootstrap required, opened a new log
I20260808 03:57:04.269423 28328 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "7627110361314a90abfa6c03debf9a5f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47222
I20260808 03:57:04.274121 28584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 837832b4e7124c42a52aed11f7ae7b63: Bootstrap starting.
I20260808 03:57:04.277118 28584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 837832b4e7124c42a52aed11f7ae7b63: Neither blocks nor log segments found. Creating new log.
I20260808 03:57:04.278095 28584 log.cc:824] T 00000000000000000000000000000000 P 837832b4e7124c42a52aed11f7ae7b63: Log is configured to *not* fsync() on all Append() calls
I20260808 03:57:04.279031 28361 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7627110361314a90abfa6c03debf9a5f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44159 } } peers { permanent_uuid: "31d63ab376764dacb7d53c495845655c" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } } peers { permanent_uuid: "837832b4e7124c42a52aed11f7ae7b63" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } }
I20260808 03:57:04.279436 28361 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7627110361314a90abfa6c03debf9a5f, State: Initialized, Role: FOLLOWER
I20260808 03:57:04.279954 28361 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7627110361314a90abfa6c03debf9a5f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44159 } } peers { permanent_uuid: "31d63ab376764dacb7d53c495845655c" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } } peers { permanent_uuid: "837832b4e7124c42a52aed11f7ae7b63" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } }
I20260808 03:57:04.281136 28584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 837832b4e7124c42a52aed11f7ae7b63: No bootstrap required, opened a new log
I20260808 03:57:04.280879 28603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7627110361314a90abfa6c03debf9a5f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44159 } } peers { permanent_uuid: "31d63ab376764dacb7d53c495845655c" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } } peers { permanent_uuid: "837832b4e7124c42a52aed11f7ae7b63" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } } }
I20260808 03:57:04.281267 28603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:57:04.282004 28361 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:57:04.282119 28328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 03:57:04.282331 28328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:57:04.285686 28328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7627110361314a90abfa6c03debf9a5f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44159 } } peers { permanent_uuid: "31d63ab376764dacb7d53c495845655c" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } } peers { permanent_uuid: "837832b4e7124c42a52aed11f7ae7b63" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } }
I20260808 03:57:04.288105 28328 leader_election.cc:288] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f [CANDIDATE]: Term 1 election: Requested vote from peers 31d63ab376764dacb7d53c495845655c (127.26.146.189:38067), 837832b4e7124c42a52aed11f7ae7b63 (127.26.146.188:36487)
I20260808 03:57:04.291311 28584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 837832b4e7124c42a52aed11f7ae7b63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7627110361314a90abfa6c03debf9a5f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44159 } } peers { permanent_uuid: "31d63ab376764dacb7d53c495845655c" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } } peers { permanent_uuid: "837832b4e7124c42a52aed11f7ae7b63" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } }
I20260808 03:57:04.291764 28584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 837832b4e7124c42a52aed11f7ae7b63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 837832b4e7124c42a52aed11f7ae7b63, State: Initialized, Role: FOLLOWER
I20260808 03:57:04.292348 28584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 837832b4e7124c42a52aed11f7ae7b63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7627110361314a90abfa6c03debf9a5f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44159 } } peers { permanent_uuid: "31d63ab376764dacb7d53c495845655c" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } } peers { permanent_uuid: "837832b4e7124c42a52aed11f7ae7b63" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } }
I20260808 03:57:04.293668 28617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 837832b4e7124c42a52aed11f7ae7b63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7627110361314a90abfa6c03debf9a5f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44159 } } peers { permanent_uuid: "31d63ab376764dacb7d53c495845655c" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } } peers { permanent_uuid: "837832b4e7124c42a52aed11f7ae7b63" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } } }
I20260808 03:57:04.294730 28617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 837832b4e7124c42a52aed11f7ae7b63 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:57:04.294834 28584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 837832b4e7124c42a52aed11f7ae7b63 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:57:04.294874 28550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7627110361314a90abfa6c03debf9a5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "837832b4e7124c42a52aed11f7ae7b63"
I20260808 03:57:04.295361 28550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 837832b4e7124c42a52aed11f7ae7b63 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:57:04.298998 28550 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 837832b4e7124c42a52aed11f7ae7b63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7627110361314a90abfa6c03debf9a5f in term 1.
I20260808 03:57:04.299768 28295 leader_election.cc:302] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7627110361314a90abfa6c03debf9a5f, 837832b4e7124c42a52aed11f7ae7b63; no voters:
I20260808 03:57:04.300335 28603 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:57:04.311340 28603 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f [term 1 LEADER]: Becoming Leader. State: Replica: 7627110361314a90abfa6c03debf9a5f, State: Running, Role: LEADER
I20260808 03:57:04.312240 28603 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7627110361314a90abfa6c03debf9a5f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44159 } } peers { permanent_uuid: "31d63ab376764dacb7d53c495845655c" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } } peers { permanent_uuid: "837832b4e7124c42a52aed11f7ae7b63" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } }
I20260808 03:57:04.320518 28623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7627110361314a90abfa6c03debf9a5f. Latest consensus state: current_term: 1 leader_uuid: "7627110361314a90abfa6c03debf9a5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7627110361314a90abfa6c03debf9a5f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44159 } } peers { permanent_uuid: "31d63ab376764dacb7d53c495845655c" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } } peers { permanent_uuid: "837832b4e7124c42a52aed11f7ae7b63" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } } }
I20260808 03:57:04.321030 28623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f [sys.catalog]: This master's current role is: LEADER
I20260808 03:57:04.322948 28640 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 03:57:04.323410 28638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:57:04.337689 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:04.305420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42318 (local address 127.26.146.189:38067)
0808 03:57:04.305593 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:04.305600 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:04.311155 (+ 5555us) server_negotiation.cc:407] Connection header received
0808 03:57:04.311704 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:04.311711 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:04.311823 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:04.311903 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:04.316591 (+ 4688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.317786 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.325321 (+ 7535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.325990 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.326111 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:04.330705 (+ 4594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:57:04.330712 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:57:04.330717 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:57:04.330762 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:57:04.334233 (+ 3471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:04.335128 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:04.335133 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:04.335136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:04.335323 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:04.335772 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:04.335778 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:04.335781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:04.336223 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:57:04.336470 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:04.337062 (+ 592us) server_negotiation.cc:299] Negotiation successful
0808 03:57:04.337306 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 03:57:04.329080 28272 master_runner.cc:186] Error getting master registration for 127.26.146.189:38067: OK, instance_id { permanent_uuid: "31d63ab376764dacb7d53c495845655c" instance_seqno: 1786161423644318 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 03:57:04.341080 28272 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 03:57:04.341434 28591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:04.316168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53378 (local address 127.26.146.188:36487)
0808 03:57:04.316357 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:04.316367 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:04.321749 (+ 5382us) server_negotiation.cc:407] Connection header received
0808 03:57:04.323120 (+ 1371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:04.323127 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:04.323241 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:04.323319 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:04.326274 (+ 2955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.327308 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.330710 (+ 3402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.331380 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.331483 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:04.334868 (+ 3385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:57:04.334875 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:57:04.334880 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:57:04.334920 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:57:04.338236 (+ 3316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:04.339025 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:04.339030 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:04.339033 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:04.339182 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:04.339638 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:04.339643 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:04.339646 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:04.340010 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:57:04.340269 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:04.340826 (+ 557us) server_negotiation.cc:299] Negotiation successful
0808 03:57:04.341075 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 03:57:04.348193 28298 tablet.cc:2369] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:57:04.348618 28594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:04.304973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47238 (local address 127.26.146.190:44159)
0808 03:57:04.305176 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:04.305182 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:04.319847 (+ 14665us) server_negotiation.cc:407] Connection header received
0808 03:57:04.320712 (+ 865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:04.320719 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:04.320817 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:04.320896 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:04.329202 (+ 8306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.330347 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.333306 (+ 2959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.334068 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.334187 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:04.341393 (+ 7206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:57:04.341400 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:57:04.341404 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:57:04.341446 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:57:04.344837 (+ 3391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:04.345621 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:04.345627 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:04.345631 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:04.345802 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:04.346243 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:04.346248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:04.346267 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:04.346629 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:57:04.346885 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:04.347364 (+ 479us) server_negotiation.cc:299] Negotiation successful
0808 03:57:04.347748 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 03:57:04.352427 28638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:57:04.358333 28483 master_runner.cc:186] Error getting master registration for 127.26.146.189:38067: OK, instance_id { permanent_uuid: "31d63ab376764dacb7d53c495845655c" instance_seqno: 1786161423644318 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 03:57:04.368562 28483 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 03:57:04.374989 28550 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 837832b4e7124c42a52aed11f7ae7b63 [term 1 FOLLOWER]: Refusing update from remote peer 7627110361314a90abfa6c03debf9a5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260808 03:57:04.375698 28654 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 837832b4e7124c42a52aed11f7ae7b63: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 03:57:04.375864 28654 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 03:57:04.376001 28623 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "837832b4e7124c42a52aed11f7ae7b63" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:57:04.387254 28617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 837832b4e7124c42a52aed11f7ae7b63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7627110361314a90abfa6c03debf9a5f. Latest consensus state: current_term: 1 leader_uuid: "7627110361314a90abfa6c03debf9a5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7627110361314a90abfa6c03debf9a5f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44159 } } peers { permanent_uuid: "31d63ab376764dacb7d53c495845655c" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } } peers { permanent_uuid: "837832b4e7124c42a52aed11f7ae7b63" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } } }
I20260808 03:57:04.387485 28617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 837832b4e7124c42a52aed11f7ae7b63 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:57:04.387612 28471 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } attempt: 5
I20260808 03:57:04.392377 28623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7627110361314a90abfa6c03debf9a5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7627110361314a90abfa6c03debf9a5f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44159 } } peers { permanent_uuid: "31d63ab376764dacb7d53c495845655c" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } } peers { permanent_uuid: "837832b4e7124c42a52aed11f7ae7b63" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } } }
I20260808 03:57:04.392654 28623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f [sys.catalog]: This master's current role is: LEADER
I20260808 03:57:04.395659 28638 catalog_manager.cc:1347] Generated new cluster ID: c9d9ce7146bd4b708a75277fd142a141
I20260808 03:57:04.395830 28638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:57:04.401960 28657 mvcc.cc:205] Tried to move back new op lower bound from 7316117194224312320 to 7316117193990057984. Current Snapshot: MvccSnapshot[applied={T|T < 7316117194224312320}]
I20260808 03:57:04.406366 28617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 837832b4e7124c42a52aed11f7ae7b63 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7627110361314a90abfa6c03debf9a5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7627110361314a90abfa6c03debf9a5f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44159 } } peers { permanent_uuid: "31d63ab376764dacb7d53c495845655c" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } } peers { permanent_uuid: "837832b4e7124c42a52aed11f7ae7b63" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } } }
Aug 08 03:57:04 dist-test-slave-jlzn krb5kdc[28257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786161423, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.146.189@KRBTEST.COM for kudu/127.26.146.188@KRBTEST.COM
I20260808 03:57:04.406615 28617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 837832b4e7124c42a52aed11f7ae7b63 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:57:04.414278 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:04.388164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.146.188:36487 (local address 127.0.0.1:53386)
0808 03:57:04.388335 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 03:57:04.388355 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:57:04.388472 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:57:04.389152 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:57:04.389157 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:57:04.389811 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:57:04.390032 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:57:04.390046 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.391548 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.391554 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:57:04.392967 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:57:04.392986 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.403277 (+ 10291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.403285 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:57:04.403534 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:04.404557 (+ 1023us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:57:04.404592 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:57:04.407332 (+ 2740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:57:04.411250 (+ 3918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:57:04.411255 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:57:04.411259 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:57:04.411778 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:57:04.412286 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:57:04.412291 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:57:04.412295 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:57:04.412496 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:57:04.413243 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:57:04.413248 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:57:04.413415 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 03:57:04.413568 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 03:57:04.413909 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 03:57:04.416154 28591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:04.388496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53386 (local address 127.26.146.188:36487)
0808 03:57:04.388683 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:04.388689 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:04.388709 (+ 20us) server_negotiation.cc:407] Connection header received
0808 03:57:04.388759 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:04.388764 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:04.388839 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:04.388939 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:04.390240 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.391386 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.393132 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.402895 (+ 9763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.403055 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:04.407552 (+ 4497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:57:04.407559 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:57:04.407564 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:57:04.407607 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:57:04.411110 (+ 3503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:04.411960 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:04.411966 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:04.411970 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:04.412141 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:57:04.412643 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:57:04.412649 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:57:04.412652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:57:04.413012 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:57:04.413178 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:04.415159 (+ 1981us) server_negotiation.cc:299] Negotiation successful
0808 03:57:04.415404 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 03:57:04.422188 28471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31d63ab376764dacb7d53c495845655c: Bootstrap starting.
I20260808 03:57:04.425944 28638 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:57:04.427762 28638 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:57:04.437763 28471 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31d63ab376764dacb7d53c495845655c: Neither blocks nor log segments found. Creating new log.
I20260808 03:57:04.441139 28471 log.cc:824] T 00000000000000000000000000000000 P 31d63ab376764dacb7d53c495845655c: Log is configured to *not* fsync() on all Append() calls
I20260808 03:57:04.449411 28638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f: Generated new TSK 0
I20260808 03:57:04.450459 28638 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:57:04.455000 28471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31d63ab376764dacb7d53c495845655c: No bootstrap required, opened a new log
I20260808 03:57:04.466349 28471 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31d63ab376764dacb7d53c495845655c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7627110361314a90abfa6c03debf9a5f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44159 } } peers { permanent_uuid: "31d63ab376764dacb7d53c495845655c" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } } peers { permanent_uuid: "837832b4e7124c42a52aed11f7ae7b63" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } }
I20260808 03:57:04.466856 28471 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31d63ab376764dacb7d53c495845655c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31d63ab376764dacb7d53c495845655c, State: Initialized, Role: FOLLOWER
I20260808 03:57:04.467430 28471 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31d63ab376764dacb7d53c495845655c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7627110361314a90abfa6c03debf9a5f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44159 } } peers { permanent_uuid: "31d63ab376764dacb7d53c495845655c" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } } peers { permanent_uuid: "837832b4e7124c42a52aed11f7ae7b63" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } }
I20260808 03:57:04.481433 28666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31d63ab376764dacb7d53c495845655c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7627110361314a90abfa6c03debf9a5f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44159 } } peers { permanent_uuid: "31d63ab376764dacb7d53c495845655c" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } } peers { permanent_uuid: "837832b4e7124c42a52aed11f7ae7b63" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } } }
I20260808 03:57:04.481998 28666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31d63ab376764dacb7d53c495845655c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:57:04.482916 28471 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31d63ab376764dacb7d53c495845655c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:57:04.483415 28428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31d63ab376764dacb7d53c495845655c [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:57:04.487332 28428 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 31d63ab376764dacb7d53c495845655c [term 1 FOLLOWER]: Refusing update from remote peer 7627110361314a90abfa6c03debf9a5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:57:04.487979 28430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7627110361314a90abfa6c03debf9a5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "31d63ab376764dacb7d53c495845655c"
I20260808 03:57:04.491752 28430 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 31d63ab376764dacb7d53c495845655c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7627110361314a90abfa6c03debf9a5f in term 1.
I20260808 03:57:04.494612 28603 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "31d63ab376764dacb7d53c495845655c" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:57:04.519935 28672 mvcc.cc:205] Tried to move back new op lower bound from 7316117194390482944 to 7316117193990057984. Current Snapshot: MvccSnapshot[applied={T|T < 7316117194224312320}]
I20260808 03:57:04.526396 28666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31d63ab376764dacb7d53c495845655c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7627110361314a90abfa6c03debf9a5f. Latest consensus state: current_term: 1 leader_uuid: "7627110361314a90abfa6c03debf9a5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7627110361314a90abfa6c03debf9a5f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44159 } } peers { permanent_uuid: "31d63ab376764dacb7d53c495845655c" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } } peers { permanent_uuid: "837832b4e7124c42a52aed11f7ae7b63" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } } }
I20260808 03:57:04.526764 28666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31d63ab376764dacb7d53c495845655c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:57:04.527168 28675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31d63ab376764dacb7d53c495845655c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7627110361314a90abfa6c03debf9a5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7627110361314a90abfa6c03debf9a5f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44159 } } peers { permanent_uuid: "31d63ab376764dacb7d53c495845655c" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } } peers { permanent_uuid: "837832b4e7124c42a52aed11f7ae7b63" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } } }
I20260808 03:57:04.527575 28675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31d63ab376764dacb7d53c495845655c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:57:04.570807 28688 catalog_manager.cc:1259] Loaded cluster ID: c9d9ce7146bd4b708a75277fd142a141
I20260808 03:57:04.570931 28688 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 31d63ab376764dacb7d53c495845655c: loading cluster ID for follower catalog manager: success
I20260808 03:57:04.574702 28688 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 31d63ab376764dacb7d53c495845655c: acquiring CA information for follower catalog manager: success
I20260808 03:57:04.577245 28688 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 31d63ab376764dacb7d53c495845655c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 03:57:04.587170 27210 mini_kdc.cc:348] Time spent kdestroy: real 0.127s user 0.000s sys 0.031s
W20260808 03:57:04.652783 28478 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:57:04.636987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42334 (local address 127.26.146.189:38067)
0808 03:57:04.637184 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:04.637192 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:04.650530 (+ 13338us) server_negotiation.cc:407] Connection header received
0808 03:57:04.651057 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:04.651063 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:04.651184 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:04.651271 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:04.652534 (+ 1263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42334: BlockingRecv error: recv got EOF from 127.0.0.1:42334 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 03:57:04.654841 28594 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:57:04.648349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47246 (local address 127.26.146.190:44159)
0808 03:57:04.648578 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:04.648585 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:04.652785 (+ 4200us) server_negotiation.cc:407] Connection header received
0808 03:57:04.652946 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:04.652951 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:04.653034 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:04.653134 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:04.654593 (+ 1459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47246: BlockingRecv error: recv got EOF from 127.0.0.1:47246 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260808 03:57:04.655689 28591 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:57:04.653474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53388 (local address 127.26.146.188:36487)
0808 03:57:04.653665 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:04.653671 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:04.653845 (+ 174us) server_negotiation.cc:407] Connection header received
0808 03:57:04.654027 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:04.654032 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:04.654113 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:04.654209 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:04.655449 (+ 1240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53388: BlockingRecv error: recv got EOF from 127.0.0.1:53388 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 03:57:04.719344 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:04.694809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42350 (local address 127.26.146.189:38067)
0808 03:57:04.695000 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:04.695006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:04.712433 (+ 17427us) server_negotiation.cc:407] Connection header received
0808 03:57:04.712693 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:04.712699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:04.712925 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:04.713117 (+ 192us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:04.713603 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.714649 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.716662 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.717264 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.717380 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:04.717962 (+ 582us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:04.718630 (+ 668us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:04.718769 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:04.718891 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 03:57:04.718982 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 03:57:04.762467 28591 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:57:04.753363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53390 (local address 127.26.146.188:36487)
0808 03:57:04.753644 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:04.753652 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:04.760019 (+ 6367us) server_negotiation.cc:407] Connection header received
0808 03:57:04.760810 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:04.760818 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:04.760922 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:04.761019 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:04.762216 (+ 1197us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53390: BlockingRecv error: recv got EOF from 127.0.0.1:53390 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 03:57:04.826822 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:04.806799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42358 (local address 127.26.146.189:38067)
0808 03:57:04.806991 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:04.806998 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:04.820183 (+ 13185us) server_negotiation.cc:407] Connection header received
0808 03:57:04.820406 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:04.820413 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:04.820529 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:04.820640 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:04.821241 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.822372 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.824527 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.825172 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.825299 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:04.825913 (+ 614us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:04.826091 (+ 178us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:04.826161 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:04.826409 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 03:57:04.826499 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260808 03:57:04.868892 28591 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:57:04.861096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53404 (local address 127.26.146.188:36487)
0808 03:57:04.861318 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:04.861324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:04.867226 (+ 5902us) server_negotiation.cc:407] Connection header received
0808 03:57:04.867456 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:04.867462 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:04.867565 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:04.867682 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:04.868616 (+ 934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53404: BlockingRecv error: recv got EOF from 127.0.0.1:53404 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 03:57:04.925956 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:04.918380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42364 (local address 127.26.146.189:38067)
0808 03:57:04.918574 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:04.918581 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:04.919171 (+ 590us) server_negotiation.cc:407] Connection header received
0808 03:57:04.919512 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:04.919519 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:04.919620 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:04.919700 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:04.920455 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.921617 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.923651 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.924367 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.924474 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:04.925079 (+ 605us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:04.925210 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:04.925281 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:04.925513 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 03:57:04.925610 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260808 03:57:04.973897 28591 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:57:04.969206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53408 (local address 127.26.146.188:36487)
0808 03:57:04.969393 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:04.969399 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:04.972366 (+ 2967us) server_negotiation.cc:407] Connection header received
0808 03:57:04.972561 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:04.972568 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:04.972673 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:04.972792 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:04.973632 (+ 840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53408: BlockingRecv error: recv got EOF from 127.0.0.1:53408 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 03:57:04.995188 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:04.989389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42380 (local address 127.26.146.189:38067)
0808 03:57:04.989571 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:04.989576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:04.990090 (+ 514us) server_negotiation.cc:407] Connection header received
0808 03:57:04.990401 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:04.990407 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:04.990491 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:04.990577 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:04.991039 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.991917 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.993270 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:04.993878 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:04.993994 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:04.994418 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:04.994554 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:04.994633 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:04.994816 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 03:57:04.994908 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260808 03:57:05.002735 28591 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:57:05.000797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53424 (local address 127.26.146.188:36487)
0808 03:57:05.001052 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:05.001058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:05.001433 (+ 375us) server_negotiation.cc:407] Connection header received
0808 03:57:05.001578 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:05.001584 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:05.001692 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:05.001808 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:05.002508 (+ 700us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53424: BlockingRecv error: recv got EOF from 127.0.0.1:53424 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
I20260808 03:57:05.019475 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:05.013659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42390 (local address 127.26.146.189:38067)
0808 03:57:05.013896 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:05.013903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:05.014334 (+ 431us) server_negotiation.cc:407] Connection header received
0808 03:57:05.014470 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:05.014476 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:05.014574 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:05.014678 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:05.015021 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:05.016157 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:05.017461 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:05.018155 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:05.018284 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:05.018643 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:05.018796 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:05.018888 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:05.019028 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 03:57:05.019114 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 03:57:05.027868 28591 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:57:05.025992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53426 (local address 127.26.146.188:36487)
0808 03:57:05.026190 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:05.026196 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:05.026520 (+ 324us) server_negotiation.cc:407] Connection header received
0808 03:57:05.026653 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:05.026658 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:05.026754 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:05.026865 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:05.027606 (+ 741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53426: BlockingRecv error: recv got EOF from 127.0.0.1:53426 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 03:57:05.053020 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:05.047057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42402 (local address 127.26.146.189:38067)
0808 03:57:05.047267 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:05.047274 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:05.047566 (+ 292us) server_negotiation.cc:407] Connection header received
0808 03:57:05.047746 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:05.047752 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:05.047888 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:05.048026 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:05.048411 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:05.049706 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:05.051079 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:05.051745 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:05.051907 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:05.052304 (+ 397us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:05.052428 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:05.052507 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:05.052645 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 03:57:05.052730 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 03:57:05.054461 28603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7627110361314a90abfa6c03debf9a5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7627110361314a90abfa6c03debf9a5f" member_type: VOTER last_known_addr { host: "127.26.146.190" port: 44159 } } peers { permanent_uuid: "31d63ab376764dacb7d53c495845655c" member_type: VOTER last_known_addr { host: "127.26.146.189" port: 38067 } } peers { permanent_uuid: "837832b4e7124c42a52aed11f7ae7b63" member_type: VOTER last_known_addr { host: "127.26.146.188" port: 36487 } } }
I20260808 03:57:05.054749 28603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7627110361314a90abfa6c03debf9a5f [sys.catalog]: This master's current role is: LEADER
W20260808 03:57:05.060842 28591 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:57:05.059002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53434 (local address 127.26.146.188:36487)
0808 03:57:05.059167 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:05.059171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:05.059575 (+ 404us) server_negotiation.cc:407] Connection header received
0808 03:57:05.059706 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:05.059712 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:05.059826 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:05.059924 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:05.060612 (+ 688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53434: BlockingRecv error: recv got EOF from 127.0.0.1:53434 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 03:57:05.100973 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:05.095874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42414 (local address 127.26.146.189:38067)
0808 03:57:05.096115 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:05.096121 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:05.096437 (+ 316us) server_negotiation.cc:407] Connection header received
0808 03:57:05.096640 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:05.096646 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:05.096735 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:05.096819 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:05.097297 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:05.097997 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:05.099328 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:05.099746 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:05.099886 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:05.100265 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:05.100381 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:05.100455 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:05.100603 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 03:57:05.100685 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 03:57:05.108014 28591 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:57:05.106154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53436 (local address 127.26.146.188:36487)
0808 03:57:05.106327 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:05.106332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:05.106676 (+ 344us) server_negotiation.cc:407] Connection header received
0808 03:57:05.106794 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:05.106798 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:05.106879 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:05.106995 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:05.107711 (+ 716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53436: BlockingRecv error: recv got EOF from 127.0.0.1:53436 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 03:57:05.180540 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:05.175328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42420 (local address 127.26.146.189:38067)
0808 03:57:05.175503 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:05.175507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:05.175876 (+ 369us) server_negotiation.cc:407] Connection header received
0808 03:57:05.176162 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:05.176168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:05.176259 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:05.176347 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:05.176803 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:05.177519 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:05.178846 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:05.179287 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:05.179386 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:05.179884 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:05.180006 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:05.180082 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:05.180251 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 03:57:05.180321 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 03:57:05.188748 28591 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:57:05.186765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53452 (local address 127.26.146.188:36487)
0808 03:57:05.186952 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:05.186956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:05.187428 (+ 472us) server_negotiation.cc:407] Connection header received
0808 03:57:05.187580 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:05.187583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:05.187658 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:05.187739 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:05.188531 (+ 792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53452: BlockingRecv error: recv got EOF from 127.0.0.1:53452 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 03:57:05.326452 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:05.321306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42428 (local address 127.26.146.189:38067)
0808 03:57:05.321524 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:05.321532 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:05.321839 (+ 307us) server_negotiation.cc:407] Connection header received
0808 03:57:05.322047 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:05.322051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:05.322153 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:05.322236 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:05.322743 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:05.323477 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:05.324965 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:05.325411 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:05.325543 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:05.325896 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:05.325996 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:05.326060 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:05.326155 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 03:57:05.326220 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 03:57:05.333774 28591 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:57:05.331981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53464 (local address 127.26.146.188:36487)
0808 03:57:05.332197 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:05.332203 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:05.332454 (+ 251us) server_negotiation.cc:407] Connection header received
0808 03:57:05.332608 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:05.332636 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:05.332712 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:05.332791 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:57:05.333500 (+ 709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53464: BlockingRecv error: recv got EOF from 127.0.0.1:53464 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 03:57:05.377867 28654 catalog_manager.cc:1259] Loaded cluster ID: c9d9ce7146bd4b708a75277fd142a141
I20260808 03:57:05.377966 28654 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 837832b4e7124c42a52aed11f7ae7b63: loading cluster ID for follower catalog manager: success
I20260808 03:57:05.380558 28654 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 837832b4e7124c42a52aed11f7ae7b63: acquiring CA information for follower catalog manager: success
I20260808 03:57:05.382355 28654 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 837832b4e7124c42a52aed11f7ae7b63: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 03:57:05.598266 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:05.593468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42434 (local address 127.26.146.189:38067)
0808 03:57:05.593718 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:05.593725 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:05.593917 (+ 192us) server_negotiation.cc:407] Connection header received
0808 03:57:05.594141 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:05.594146 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:05.594218 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:05.594299 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:05.594694 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:05.595497 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:05.596880 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:05.597286 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:05.597379 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:05.597679 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:05.597776 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:05.597834 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:05.597929 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 03:57:05.597989 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 03:57:05.609404 28591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:05.604087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53472 (local address 127.26.146.188:36487)
0808 03:57:05.604282 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:05.604288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:05.604427 (+ 139us) server_negotiation.cc:407] Connection header received
0808 03:57:05.604683 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:05.604689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:05.604850 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:05.604950 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:05.605362 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:05.606061 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:05.607325 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:05.607695 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:05.607821 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:05.608304 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:05.608820 (+ 516us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:05.608976 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:05.609040 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 03:57:05.609120 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 03:57:05.621732 28861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:05.614758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47252 (local address 127.26.146.190:44159)
0808 03:57:05.615371 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:05.615376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:05.615397 (+ 21us) server_negotiation.cc:407] Connection header received
0808 03:57:05.615547 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:05.615551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:05.615770 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:05.616007 (+ 237us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:05.616533 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:05.617590 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:05.619371 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:05.620063 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:05.620192 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:05.620745 (+ 553us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:05.621157 (+ 412us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:05.621268 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:05.621381 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 03:57:05.621447 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":278,"threads_started":1}
I20260808 03:57:05.622632 28591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:05.615377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53474 (local address 127.26.146.188:36487)
0808 03:57:05.615567 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:05.615572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:05.616482 (+ 910us) server_negotiation.cc:407] Connection header received
0808 03:57:05.616633 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:05.616636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:05.616733 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:05.616817 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:05.617301 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:05.618159 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:05.620230 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:05.620813 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:05.620901 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:05.621663 (+ 762us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:05.621778 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:05.621864 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:05.622279 (+ 415us) server_negotiation.cc:299] Negotiation successful
0808 03:57:05.622352 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 03:57:05.623271 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:57:05.614749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42438 (local address 127.26.146.189:38067)
0808 03:57:05.614925 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 03:57:05.614930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:57:05.616724 (+ 1794us) server_negotiation.cc:407] Connection header received
0808 03:57:05.616917 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:57:05.616923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:57:05.617008 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:57:05.617081 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:57:05.617808 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:05.618643 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:05.621214 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:57:05.621839 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:57:05.621946 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:57:05.622474 (+ 528us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:57:05.622591 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:57:05.622668 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 03:57:05.622900 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 03:57:05.622979 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 03:57:05.626353 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 28272
I20260808 03:57:05.627341 28355 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:57:05.743779 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 28272
I20260808 03:57:05.772114 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 28371
I20260808 03:57:05.773296 28464 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:57:05.879921 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 28371
I20260808 03:57:05.904412 27210 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu pid 28483
I20260808 03:57:05.905660 28578 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:57:06.011590 27210 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT_CS6X/build/asan/bin/kudu with pid 28483
2026-08-08T03:57:06Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4882 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9452 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (367868 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST